Master control device and methods therefor

ABSTRACT

Implementations relate to a master control device. In some implementations, a master control device includes a control body comprising a proximal end, a thumb grip portion, and a finger grip portion. The control body has a length configured to engage the proximal end of the control body in the palm of a hand of the user while the hand engages the thumb grip portion with a thumb of the hand and the finger grip portion with a finger of the hand. The control body is configured to allow the proximal end of the control body to be selectively engaged by the palm of the hand and selectively disengaged from the palm of the hand while the hand engages the thumb grip portion with the thumb of the hand and the finger grip portion with the finger of the hand.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional PatentApplication No. 62/586,752, filed Nov. 15, 2017 and titled “MasterControl Device and Methods Therefor,” the entire contents of which arehereby incorporated by reference.

BACKGROUND

In teleoperated operations such as teleoperated surgery, a usertypically operates a master controller, e.g., included in a workstationor console, to remotely control (e.g., teleoperate) the motion andfunctions of instruments at a work site (e.g., surgical site). Themaster controller utilizes master controls, which will typically includeone or more hand input devices such as pincher grips, joysticks,exo-skeletal gloves, or the like. These hand input devices are incommunication with the controlled instrument. More specifically, amanipulator or “slave” device including the instrument is moved based onthe user's manipulation of the hand input devices. In some examples of asurgical or other medical operation, a hand input device may control,via the teleoperated surgery system, a variety of surgical instrumentssuch as tissue graspers, needle drivers, electrosurgical cautery probes,cameras, etc. Each of these instruments performs functions for thesurgeon, for example, holding or driving a needle, grasping a bloodvessel, or dissecting, cauterizing, or coagulating tissue.

For some hand input devices, the user may have difficulty manipulating ahand input device while maintaining a secure grip on the hand inputdevice. Further, in some situations, it may be beneficial to operate thehand input device without being bound to a stationary workstation orconsole.

SUMMARY

Implementations of the present application relate to a master controldevice and methods for using such a control device. In someimplementations, a master control device includes a control bodycomprising a proximal end, a thumb grip portion coupled to the controlbody, and a finger grip portion coupled to the control body. The controlbody has a length configured to engage the proximal end of the controlbody in the palm of a hand of the user while the hand engages the thumbgrip portion with a thumb of the hand and the finger grip portion with afinger of the hand. The control body is configured to allow the proximalend of the control body to be selectively engaged by the palm of thehand and selectively disengaged from the palm of the hand while the handengages the thumb grip portion with the thumb of the hand and the fingergrip portion with the finger of the hand.

Various implementations and examples of the master control device aredescribed. For example, in some implementations, the master controldevice is a surgical system master control device configured to providecontrol signals to a surgical teleoperated system. In someimplementations, the master control device includes a sensor configuredto detect at least one of a position and an orientation of the mastercontrol device in a working environment of the master control device. Insome implementations, the thumb grip portion includes a thumb gripmember rotatably coupled to the control body, and the finger gripportion includes a finger grip member rotatably coupled to the controlbody. In various implementations, the master control device ismechanically ungrounded, or the control body is coupled to amechanically grounded linkage.

In various implementations, the proximal end includes an extensionmember that is rotatable about a longitudinal axis of the control bodyindependently of the control body, the thumb grip portion, and thefinger grip portion, and the master control device further includes atethered connection coupling the control body to a master controlsystem, where the tethered connection extends from the extension memberradially from a central axis of the control body, allowing a connectionpoint between the tethered connection and the extension member to rotatewith respect to the control body. In some implementations, the mastercontrol device includes a wireless transmitter configured to sendwireless signals to a master control system based on motion of the thumbgrip portion and the finger grip portion (and/or manipulation of inputcontrols).

In some implementations, the control body is configured to allow theproximal end of the control body to be moved by manipulation of thethumb and the finger of the hand on the control body to move theproximal end of the control body into selective engagement with thepalm. In some implementations, the control body is configured to allowthe proximal end of the control body to be moved between the thumb ofthe hand and an index finger of the hand while the hand engages thethumb grip portion with the thumb of the hand and the finger gripportion with the finger of the hand.

In some implementations, the control body has a shape configured toengage the proximal end of the control body in the palm of a hand of theuser while the hand pinches the thumb grip portion with a thumb of thehand and the finger grip portion with a finger of the hand. In someexamples, the proximal end includes an extension member that includes atleast a portion of a spherical surface. In various implementations, theproximal end has an axisymmetric shape with respect to the longitudinalaxis of the control body, or has an asymmetric shape with respect to thelongitudinal axis of the control body. In some examples, the proximalend includes an extension member that extends asymmetrically to one sideof a longitudinal axis of the control body, where the extension memberis receptive to grasping by a portion of the hand and/or one or morefingers during operation of the master control device. In furtherexamples, the extension member includes a finger aperture receptive tothe finger of the hand.

In some implementations, the proximal end includes an extension memberthat is rotatable about a longitudinal axis of the control bodyindependently of the control body, the thumb grip portion, and thefinger grip portion. In some implementations, the finger grip portion isconfigured to be receptive to at least one of a first finger, a secondfinger, and a third finger of the hand, and the proximal end isconfigured to be receptive to at least one of the third finger, a fourthfinger, and a fifth finger of the hand.

In some implementations, the proximal end includes an extension memberthat is translatable along a longitudinal axis of the control bodyindependently of the control body, the thumb grip portion, and thefinger grip portion. In some implementations, the proximal end includesan extension member, and the master control device further includes aswitch including a ring centered on a longitudinal axis of the controlbody and positioned between the extension member and the control body,where the ring is linearly translatable with respect to the extensionmember and with respect to the control body to activate the switch.

Some implementations further comprise an input control coupled to theproximal end and configured to detect a threshold amount of contact witha finger of the hand. In some examples, the input control is coupled toa portion of the proximal end extending asymmetrically to one side of alongitudinal axis of the control body. Some implementations furtherinclude a first sensor and a second sensor, the first sensor is coupledto the proximal end and the second sensor coupled to a different portionof the master control device, where the first sensor and second sensorare configured to sense different portions of the hand, e.g., to sense aparticular grasping configuration of the hand with the master controldevice. In some implementations, a distal weighted element positioned atthe distal end of the control body and a proximal weighted elementpositioned at the proximal end of the control body are weighted toprovide a center of gravity between a respective finger contact surfaceof the thumb grip portion and the finger grip portion.

A master control system includes a master device that includes a controlbody comprising a proximal end, a thumb grip portion coupled to thecontrol body, and a finger grip portion coupled to the control body. Thecontrol body has a length configured to, in a first position, engage theproximal end of the control body in the palm of a hand of the user whilethe hand engages the thumb grip portion with a thumb of the hand and thefinger grip portion with a finger of the hand. The control body isconfigured to allow the proximal end of the control body to beselectively moved to a second position that is disengaged from the palmof the hand, while the hand engages the thumb grip portion with thethumb of the hand and the finger grip portion with the finger of thehand. The system also includes a controller coupled to a slave deviceand in communication with the master device, where the controller isconfigured to provide control signals to the slave surgical device whilea master-slave control relationship is provided between the masterdevice and the slave device.

Various implementations and examples of the system are described. Forexample, in some implementations, the master control system is asurgical master control system, and the slave device is a surgical slavedevice including a surgical instrument. In some implementations, themaster control system is configured to maintain the master-slave controlrelationship while the user performs a first movement of the masterdevice from the first position to the second position. In someimplementations, the control body is configured to allow the proximalend of the control body to be moved to the second position that isbetween the thumb of the hand and an index finger of the hand, while thehand engages the thumb grip portion with the thumb of the hand and thefinger grip portion with the finger of the hand. In someimplementations, the control body has a shape configured to engage(e.g., ground) the proximal end of the control body in the palm of ahand of the user while the hand pinches the thumb grip portion with athumb of the hand and the finger grip portion with a finger of the hand.

In various implementations, the proximal end includes an extensionmember that has an axisymmetric shape with respect to a longitudinalaxis of the control body, or that has an asymmetric shape with respectto the longitudinal axis of the control body. In various examples, theextension member can include a finger aperture receptive to a finger ofthe hand. Some implementations include an extension member that isrotatable about a longitudinal axis of the control body independently ofthe control body, the thumb grip portion, and the finger grip portion.

In some examples, the proximal end includes an extension member, and themaster control system further includes a switch including a ringcentered on a longitudinal axis of the control body and positionedbetween the extension member and the control body, where the ring islinearly translatable with respect to the extension member and withrespect to the control body to activate the switch. In someimplementations, an input control is coupled to a portion of theproximal end extending asymmetrically to one side of a longitudinal axisof the control body. The master device can be mechanically ungrounded orcan be mechanically grounded.

In some implementations, a method of operating a teleoperated systemincludes establishing a master-slave control relationship between amaster device and a slave instrument. The master device comprises acontrol body having a proximal end, a thumb grip portion, and a fingergrip portion. The method includes maintaining the control relationshipwhile the user performs a first movement of the master device. The firstmovement includes moving the master device from a first position inwhich the proximal end of the control body is engaged in the palm of ahand of the user while engaging the thumb grip portion with the thumb ofthe hand and the finger grip portion with a finger of the hand, to asecond position in which the proximal end of the control body is movedand disengaged from the palm of the hand of the user while engaging thethumb grip portion with the thumb of the hand and the finger gripportion with the finger of the hand.

Various implementations and examples of the method are described. Forexample, in some implementations, the first movement includes moving themaster device to the second position such that the proximal end of thecontrol body passes between the thumb of the hand and the index fingerof the hand while engaging the thumb grip portion with the thumb of thehand and the finger grip portion with the finger of the hand. In someimplementations, the method further includes maintaining the controlrelationship while the user performs a second movement of the masterdevice, where the second movement comprises moving the master devicefrom the second position to the first position. In some implementations,maintaining the control relationship includes sensing the moving of themaster device from the first position to the second position with asensor, and causing output of control signals indicative of the movingof the master device. The control signals cause movement of the slaveinstrument based on the moving of the master device from the firstposition to the second position. In some implementations, the proximalend includes a first sensor and a different portion of the master deviceincludes a second sensor, and further comprising sensing a presence ofthe hand relative to the master device by both the first sensor and thesecond sensor, wherein establishing the master-slave controlrelationship is performed in response to sensing the presence of thehand by both the first sensor and the second sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic view of an example teleoperated surgicalsystem, according to some implementations;

FIG. 2 is a perspective view of an example of a master controller,according to some implementations;

FIG. 3 is a perspective view of the master controller of FIG. 2 beingmanipulated by a user's hand, according to some implementations;

FIG. 4 is a perspective view of the master controller of FIG. 2 beingmanipulated by a user's hand to activate a switch adjacent to theextension member, according to some implementations;

FIG. 5 is another perspective view of the master controller of FIG. 2being manipulated by a user's hand to activate a switch adjacent to theextension member, according to some implementations.

FIG. 6 is a perspective view of the master controller of FIG. 2 beingmanipulated by a user's hand to a first orientation, according to someimplementations

FIG. 7 is a perspective view of the master controller of FIG. 2 beingmanipulated by a user's hand to a second orientation, according to someimplementations;

FIG. 8 is a perspective view of another implementation of a master handcontroller, according to some implementations;

FIG. 9 is a perspective view of another implementation of a master handcontroller, according to some implementations;

FIG. 10 is a perspective view of another example implementation of anextension member for the master hand controller, according to someimplementations;

FIG. 11 is a top plan view of another example implementation of a masterhand controller, according to some implementations;

FIG. 12 is a perspective view of another example implementation of amaster hand controller, according to some implementations;

FIG. 13 is a perspective view of another example of a master handcontroller that includes an extension member having one or more fingerapertures, according to some implementations;

FIG. 14 is a perspective view of another example of a master handcontroller that includes an extension member having a finger aperture,according to some implementations;

FIG. 15 is a perspective view of another example of a master handcontroller that includes an extension member having one or more fingerapertures, according to some implementations;

FIG. 16 is a perspective view of a user's hand holding and operating amaster hand controller of FIG. 15, according to some implementations;

FIG. 17 is a schematic illustration of view of an example controllersystem that is mechanically grounded;

FIG. 18 is a perspective view of an example control portion that ismechanically grounded and can be engaged by a user;

FIG. 19 is a flow diagram illustrating an example method for employing ahand controller including one or more features described herein,according to some implementations;

FIG. 20 is a diagrammatic illustration of an example teleoperated slavedevice and patient site, according to some implementations; and

FIG. 21 is a block diagram of an example master-slave system, which canbe used for one or more implementations described herein.

DETAILED DESCRIPTION

Implementations relate to a master control device (e.g., a “master handcontroller,” “master controller,” or “hand controller”). As described inmore detail herein, implementations provide a master controller enablinguser control over multiple functions of a system, such as a teleoperatedsystem (e.g., teleoperated surgical system). The master controller canbe adapted to mechanically ungrounded operation by a user in a standingor sitting position, e.g., close to a patient or other site ofoperations. In some implementations, the master controller may be usedin mechanically grounded operation. Functions activated at theactivation positions can include functions of surgical tools and otherinstruments used in treating patients, including instruments used inteleoperated systems.

Described features of the master controller include a proximal end ofthe controller that is configured to allow a user to manipulate themaster controller including engaging (e.g., pinching) a thumb gripportion and finger grip portion with the user's thumb and other fingerwhile the proximal end is engaged (e.g., grounded) in the palm of theuser. The control body is configured to allow the proximal end of thecontrol body to be selectively engaged by the palm of the hand andselectively disengaged from the palm of the hand. For example, thecontroller manipulation is enabled while the proximal end is moved outof engagement with the palm, e.g., moved between the thumb and finger ofthe hand. Such features allow the master controller to be moved andoriented in space with reduced restrictions to movement while enablingthe user to grasp and contact the controller more securely and causingless fatigue, thus reducing inadvertent slippage or dropping of thecontroller by the user.

Various described features of the master controller include an extensionmember at the proximal end of the master controller that rotates in oneor more directions with respect to the other portions of the mastercontroller to enable more flexible manipulation of the master controllerin space. Features include different shapes and sizes of the extensionmember that enable different amounts and/or types of engagement with theuser's hand and fingers. Customization of the extension member canprovide different lengths and shapes of the proximal end of the mastercontroller. Input controls can be provided on the extension member toenable the user to activate the input controls to activate associatedfunctions of the teleoperated system.

Described features provide various benefits. For example, a mechanicallyungrounded hand controller described herein can be provided with controlover operation and functions of a slave device, such as a surgical slavedevice. Users such as surgeons or other operators may use mastercontrollers over long periods of time during control procedures.Mechanically grounded master controllers may be used in such procedureswith reduced fatigue because the grounded connection supports the weightof the controller and may provide gravity compensation. Ungroundedmaster controllers, however, do not have this grounded connection, andthus an operator may become more fatigued in use of the controller overthe duration of a surgical procedure. Furthermore, some ungroundedmaster controllers may have tethered connections (cables, etc.) thatobstruct movement of or add weight to the controller. In addition,ungrounded master controllers (or their tethered connections) maysometimes be knocked or otherwise impacted by the operator's other hand,another person, etc. These factors may cause an ungrounded mastercontroller to slip in the hand of the user or drop out of the hand,which may cause inadvertent and dangerous movements of a controlledslave device. Furthermore, some mechanically grounded master controllersmay have similar or other issues with slippage out of an operating hand,e.g., due to blocking structures within the working environment,unexpected collisions with objects, forces applied to the mastercontroller, etc.

Features described herein provide accurate, secure, and safemanipulation of system functions using a master controller. Featuressuch as an extension member at the proximal end of the hand controllerprovide additional security and reduced fatigue in operating the mastercontroller to reduce incidences of inadvertent slippage or dropping ofthe controller by the user during controller operation. For example, theextension member is provided at a particular length and/or shape andwith a particular surface that allow the proximal end of the controllerto be readily grasped and contacted by the palm of the user's hand. Forexample, the extension member can be grasped by fingers of the user iffinger grips of the controller fall from of the user's fingers.Described features also allow the controller to have large fingertiprange of motion to provide accurate and precise control over slaveinstruments, without significantly restricting the range of controllermotion. In some implementations, a user can utilize a larger portion oftheir hand in grasping the master controller, e.g., by using additionalfingers to contact the controller in addition to two fingers contactingpincher grips, and/or by using a palm of the hand to engage thecontroller at particular times. Features such as the length, shape,grips, input controls, and other features of an extension member of thecontroller enable additional grasping security and enhanced manipulationof the controller. The described features that increase graspingsecurity, reduce fatigue, and increase accuracy of control of thecontroller are of high importance in procedures where accuracy andconsistency in instrument control are required, e.g., medical proceduresin which controlled surgical instruments operate on a live patient.

Various terms including “linear,” “center,” “parallel,” “perpendicular,”“aligned,” or particular measurements or other units as used herein canbe approximate, need not be exact, and can include typical engineeringtolerances.

Some implementations herein may relate to various instruments andportions of instruments in terms of their state in three-dimensionalspace. As used herein, the term “position” refers to the location of anobject or a portion of an object in a three dimensional space (e.g.,three degrees of translational freedom along Cartesian X, Y, Zcoordinates). As used herein, the term “orientation” refers to therotational placement of an object or a portion of an object (threedegrees of rotational freedom—e.g., roll, pitch, and yaw around theCartesian X, Y, and Z axes). As used herein, the term “pose” refers tothe position of an object or a portion of an object in at least onedegree of translational freedom and to the orientation of that object orportion of the object in at least one degree of rotational freedom (upto six total degrees of freedom).

As used herein, a mechanically ungrounded master control device refersto a master controller that is unconstrained with respect to possibleposition and orientation motion in a large working environment (e.g., anoperating area or room) and is kinematically separated from the ground,e.g., not mechanically supported by a console, supports, or other objectattached to the ground. In some implementations, a mechanicallyungrounded master control device may be in tethered or untetheredconnection with one or more associated components such as controlprocessors, data sources, sensors, power supplies, etc. For example, themaster control device may be tethered, e.g., connected physically tothese components via a cable or wire, or untethered, e.g., notphysically connected to such components and in communication with thecomponents via wireless communication signals.

Aspects of this invention augment the control capability of acomputer-assisted teleoperated system through the use of one or moremaster controllers (e.g., one, two, three, or more) for providinginstrument control in various procedures (surgical, procedures inextreme environments, or other procedures), instruction, supervision,proctoring, and other feedback to a user of the system. In some exampleimplementations, master controllers may provide control of one or moreof the operational surgical tools in the surgical environment or proxysurgical tools in a virtual environment. One example of a medical devicesystem that may incorporate one or more of these master controllers(e.g., mechanically ungrounded or mechanically grounded) is the daVinci® minimally invasive teleoperated medical system commercialized byIntuitive Surgical, Inc. of Sunnyvale, Calif.

FIG. 1 is a diagrammatic view of an example teleoperated surgical system100, including one or more master control devices, according to someimplementations.

As shown, the teleoperated surgical system 100 generally includes ateleoperated slave device 102 mounted to or near an operating table 104(e.g., table, bed, or other support) on which a patient 106 ispositioned. The teleoperated slave device 102 includes one or moremanipulator arms 108, each coupled to an instrument assembly 109. Aninstrument assembly 109 may include, for example, instruments 110. Insome examples, instruments 110 may include surgical instruments orsurgical tools. In some implementations, a surgical instrument caninclude a surgical end effector at its distal end, e.g., for treatingtissue of the patient. In various implementations, surgical instrumentscan include cameras, e.g., cameras for use with surgical procedures.Some examples of an arm assembly for the teleoperated slave device 102are shown in FIG. 20.

The teleoperated surgical system 100 includes an ungrounded mastercontroller system 120. In this example, master controller system 120includes one or more mechanically ungrounded master control devices 122(“master controllers”), some implementations of which are describedbelow, for use by a user 124. The master control device 122 includes atleast one mechanically ungrounded, unpowered master tool, e.g., handcontroller, contacted or grasped by hand of the user 124. In someimplementations, two or more mechanically ungrounded unpowered mastertools can be used, e.g., one tool used by each hand of user 124. Exampleimplementations of a master control device 122 are described in moredetail below. The master control device 122 can be operated in a sterilesurgical field close to patient, as described below. An ergonomicsupport 123 (e.g., forearm rest) may be provided in the sterile surgicalfield to support the user's forearms or elbows as the user 124manipulates master control device 122, e.g., during a surgicalprocedure.

In some implementations, the slave manipulator arms 108 and/orinstrument systems 109 may be controlled to move and articulate theinstruments 110 in response to manipulation of master control device 122by the user 124, so that the user 124 can direct surgical procedures atinternal surgical sites through minimally invasive surgical apertures.For example, one or more actuators coupled to the manipulator arms 108and/or instrument systems 109 may output force to cause links or otherportions of the arms 108 and/or instruments 110 to move in particulardegrees of freedom in response to control signals received from themaster control device 122.

The number of teleoperated surgical instruments 110 used at one time,and/or the number of arms 108 used in slave device 102, may depend onthe medical procedure to be performed and the space constraints withinthe operating room, among other factors. If it is necessary to changeone or more of the surgical instruments being used during a procedure,an assistant 128 may remove a surgical instrument no longer being usedfrom its arm 108 or instrument assembly 109 and replace that surgicalinstrument with another surgical instrument from a tray in the operatingroom.

Some implementations of the teleoperated surgical system 100 can providedifferent modes of operation. In some examples, in a non-controllingmode (e.g., safe mode) of the teleoperated surgical system 100, thecontrolled motion of the teleoperated slave device 102 is disconnectedfrom the master control device 122 in disconnected configuration, suchthat movement and other manipulation of the master control device 122does not cause motion of the teleoperated slave device 102. In acontrolling mode of the teleoperated system 100 (e.g., following mode),motion of the teleoperated slave device 102 can be controlled by themaster control device 122 such that movement and other manipulation ofthe master control device 122 causes motion of the teleoperated slavedevice 102, e.g., during a surgical procedure. Some examples of suchmodes are described in greater detail below.

In this example, user 124 may be a surgeon controlling the movement ofinstrument systems 108 or a proctor providing supervision and/orinstruction for a different surgeon or user (e.g., user 142). Eachmanipulator arm 108 and the teleoperated instrument assembly 109controlled by that manipulator may be controllably coupled to anddecoupled from mechanically ungrounded master control devices 122. Forexample, user 124 may sit or stand at the side of patient 106 whileworking in a sterile surgical field and view display device 126 during asurgical procedure. User 124 performs a medical procedure bymanipulating at least master control device 122. In some examples, user124 grasps master control device 122 in configurations described hereinso that targeting and grasping involve intuitive pointing and pinchingmotions. As the user 124 moves master control device 122, sensed spatialinformation and sensed orientation information is provided to controlsystem 110 based on the movement of master control device 122.

In some implementations, a hand-tracking transceiver 130 can be includedin the ungrounded master controller system 120. For example,hand-tracking transceiver 130 can be positioned to generate a field, forexample an electromagnetic field, an optical field (e.g., light beams),etc., in proximity to the user 124. The movement of master controldevice 122 in this field provides sensed spatial position andorientation information in a three-dimensional coordinate system, e.g.,sensed by the transceiver 130 and/or other sensors (e.g., sensorspositioned at other locations of the working volume). In some examples,the transceiver 130 can be or include an electromagnetic spatialtracking system, an inertial spatial tracking system, an optical spatialtracking system, a sonic spatial tracking system, etc. The device thatsenses and outputs sensed information may vary depending on theparticular spatial tracking system or combination of tracking systemsused. In each implementation, at least sensed position and orientationinformation for a master control device 122 is provided to a controlsystem 150.

In some implementations, the ungrounded master controller system 120also includes a display device 126. In some implementations, imagescaptured by one or more cameras of the teleoperated slave device 102(e.g., on an instrument assembly 109) can be transmitted to the displaydevice 126 and/or transmitted to one or more other displays, e.g., adisplay coupled to the teleoperated slave device 102 (not shown), adisplay of the operator input system 140, etc. For example, a surgicalenvironment near or within the patient 106 and the real or virtualinstruments controlled by the ungrounded master control device 122 canbe displayed by the display device 126 and viewed by the user 122 whilethe user is operating the ungrounded master controller system 120.Display device 126 can provide a two dimensional image 127 and/or athree-dimensional image 127 of, for example, an end effector of a slavesurgical instrument 110 and the surgical site. In some examples, displaydevice 126 provides an output that the user perceives as athree-dimensional image that includes an image 127 of an end effector ofa slave surgical instrument 110 and the surgical site. The end effectoris located within a sterile surgical field. The three-dimensional imageprovides three-dimensional depth cues to permit user 124 to assessrelative depths of instruments and patient anatomy. Thethree-dimensional depth cues permit user 124 to use visual feedback tosteer the end effector of slave surgical instrument 110 using mastercontrol device 122 to precisely target features.

Various embodiments of an ungrounded master control device are disclosedin U.S. Pat. No. 8,521,331 B1 (issued on Aug. 27, 2013, titled“Patient-side Surgeon Interface For a Minimally Invasive, TeleoperatedSurgical Instrument”), which is incorporated herein by reference in itsentirety.

In some implementations, ungrounded master controller system 120 has atleast one component within a sterile surgical field of the surgery. Thesterile surgical field is a non-contaminant zone or space near thesurgical site in which contaminants are reduced to reduce potentialbacterial (or other) contamination to the surgical site during surgery.During surgery, the distal end of at least one teleoperated surgicalinstrument 110 is positioned within a sterile surgical field. In someimplementations, the one or more components in the sterile field caninclude the master control device(s) 122. For example, master controldevice 122 is either sterile or draped so that master control device 122may be safely positioned and used within a sterile surgical field forthe surgery. This feature in combination with an image on display device126 allows a user 124 to control teleoperated slave surgical instruments110 from within the sterile surgical field. Thus, ungrounded mastercontroller system 120 permits a user 124 to work within the sterilesurgical field adjacent a patient 106 undergoing surgery.

Controlling minimally invasive slave surgical instruments 110 fromwithin the sterile surgical field permits minimally invasive surgerycombined with direct visualization of patient 106, teleoperated slavedevice 102, any manually operated surgical instruments, other machinesand/or instruments being used in the surgery, etc., by user 124. In someexamples, the proximity to patient 106 allows user 124 to control an endeffector of teleoperated slave surgical instrument 110 together with oneor more manually controlled instruments, such as a laparoscopicinstrument or a stapler.

Ungrounded master controller system 120 can reduce operating room floorrequirements for the teleoperated surgical system 100. Ungrounded mastercontroller system 120 may provide a lower-cost alternative to a groundedinput system 140 (e.g., surgeon's console 141) in a conventionalminimally invasive, teleoperated surgical system. For example,ungrounded master controller system 120 can improve safety by allowinguser 124, who is performing the operation, to directly observe patient106 and teleoperated slave device 102 while manipulating instruments110. System 120 also allows the single user 124 to operate in thesterile surgical field and perform procedures which require coordinateduse of manual surgical instruments and one or more teleoperated slavesurgical instruments. System 120 promotes collaborative procedureswithout requiring additional large stand-alone surgeon consoles. In someimplementations, assistant 128 may share system 120 to operate othersurgical instruments. In addition, multiple users (e.g., surgeons) maycollaborate using a common display device 126.

In some implementations, the teleoperated surgical system 100 may alsoinclude an grounded input system 140, which allows a second user 142(e.g., a surgeon or other type of clinician) to view images of orrepresenting the worksite and to control the operation of themanipulator arms 108 and/or the instrument assemblies 109. In someimplementations, the grounded input system 140 may be located at aconsole 141, e.g., a surgeon console, which can be located in the sameroom as operating table 104. In various implementations, the user 142can be located in a different room or a completely different buildingfrom the patient 106. For example, the surgeon console 141 can belocated outside the sterile surgical field.

In this example teleoperated system 100, grounded input system 140includes one or more mechanically grounded master control device(s)(“master controllers”) for controlling the manipulator arms 108 and theinstrument assemblies 109. The grounded master controllers may includeone or more of any number of a variety of coupled input devices, such askinematically linked (mechanically ungrounded) hand grips, joysticks,trackballs, data gloves, trigger-guns, hand-operated controllers, voicerecognition devices, touch screens, body motion or presence sensors, andthe like. In some implementations, the grounded master controllers areprovided with the same degrees of freedom as the instruments of theteleoperated assembly to provide the operator with telepresence, theperception that the master controllers are integral with the instrumentsso that the operator has a strong sense of directly controllinginstruments as if present at the worksite. In other implementations, themaster controllers may have more or fewer degrees of freedom than theassociated instruments and still provide the operator with telepresence.In some implementations, the master controllers are manual input deviceswhich move in all six Cartesian degrees of freedom, and which may alsoinclude an actuatable handle for actuating instruments (for example, forclosing grasping jaws, applying an electrical potential to an electrode,delivering a medicinal treatment, and the like). Such a grip function isan additional mechanical degree of freedom (i.e., a grip DOF). In someexamples, each manipulator arm 108 and the teleoperated instrumentsystem controlled by that manipulator arm may be controllably coupled toand decoupled from the master controllers of input system 140. In someimplementations, the grounded master controllers of the input system 140can include one or more features of hand controllers as described inimplementations herein.

The teleoperated surgical system 100 also includes a control system 150.The control system 150 includes at least one memory and at least oneprocessor (not shown), and typically a plurality of processors, foreffecting control between the teleoperated slave device 102, theungrounded master control system 120, and the grounded input system 140.The control system 150 also includes programmed instructions (e.g., acomputer-readable medium storing the instructions) to implement some orall of appropriate operations and blocks of methods in accordance withaspects disclosed herein.

For example, control system 150 maps sensed spatial motion data andsensed orientation data describing the master control device 122 inspace to a common reference frame. Control system 150 may process themapped data and generate commands to appropriately position aninstrument 110, e.g., an end effector or tip, of teleoperated slavedevice 102 based on the movement (e.g., change of position and/ororientation) of master control device 122. Control system 150 can use ateleoperation servo control system to translate and to transfer thesensed motion of master control device 122 to an associated arm 108 ofthe teleoperated slave device 102 through control commands so that user124 can manipulate the instruments 110 of the teleoperated slave device102. Control system 150 can similarly generate commands based onactivation or manipulation of input controls of the master controldevice 122 to perform other functions of the slave device 102 and orinstruments 110, e.g., move jaws of an instrument end effector, activatea cutting tool or output energy, activate a suction or irrigationfunction, etc.

While control system 150 is shown as a single block in FIG. 1, thesystem may include two or more data processing circuits with one portionof the processing optionally being performed on or adjacent theteleoperated slave device 102, another portion of the processing beingperformed at the ungrounded master controller system 120, anotherportion of the processing being performed at the grounded input system140, etc. Any of a wide variety of centralized or distributed dataprocessing architectures may be employed. Similarly, the programmedinstructions may be implemented as a number of separate programs orsubroutines, or they may be integrated into a number of other aspects ofthe teleoperated systems described herein. In one embodiment, controlsystem 110 supports one or more wireless communication protocols such asBluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and Wireless Telemetry.

In some implementations, user 124, from within the sterile surgicalfield, can control at least one proxy visual to a proctor surgeon 142 atsurgeon console 141. For example, the proxy visual is visible both indisplay device 126 and in a display device viewed in surgeon console141. Using master control device 122, user 124 can manipulate the proxyvisual of a surgical instrument to demonstrate control and use ofteleoperated slave surgical instruments 110 while user 142 uses mastercontrollers of the surgeon console 141 to control a teleoperated slaveinstrument 110. Alternatively, second user 142 can control the proxyvisual, using a master controller on the surgeon console 141, toinstruct user 124. In some implementations, user 124 can telestrate(e.g., draw a freehand sketch over a moving or still video image), orcan control a virtual hand or other pointer in the display. In someimplementations, user 124 can demonstrate how to manipulate a mastertool grip on the surgeon console 140 by manipulating a virtual image ofmaster tool grip that is presented in the displays 126 and on console140. To facilitate proctoring, a proxy visual module (not shown) of thecontroller 110 can be processed as part of a vision processingsubsystem. For example, the executing module receives position andorientation information, input control states (e.g., switch states,variable slider state, etc.), presence states, grip state, or otherinformation from the master control device 122 and renders stereoimages, which are composited with the endoscopic camera images in realtime and displayed on any combination of surgeon console 141, displaydevice 126, or any other display systems in the surgical environment.

In some implementations, a controlled teleoperated slave device 102 canbe a virtual representation of a device, e.g., presented in a graphicaltraining simulation provided by a computing device coupled to theteleoperated surgical system 100. For example, a user can manipulatemaster hand controller devices to control a displayed representation ofan end effector in virtual space of the simulation, similarly as if theend effector were a physical object coupled to a physical slave device.Some implementations can use master hand controller devices in training,e.g., demonstrate the use of instruments and controls of a workstationincluding controller devices.

In some implementations, non-teleoperated systems can also use one ormore features of the master control devices as described herein. Forexample, various types of control systems and devices, peripherals, etc.can be used with described master controllers.

Some implementations can include one or more components of ateleoperated medical system such as a da Vinci® Surgical System (e.g., aModel IS3000 or IS4000, marketed as the da Vinci® Si® or da Vinci® Xi®Surgical System), commercialized by Intuitive Surgical, Inc. ofSunnyvale, Calif. Features disclosed herein may be implemented invarious ways, including teleoperated and, if applicable,non-teleoperated (e.g., locally-controlled) implementations.Implementations on da Vinci® Surgical Systems are merely examples andare not to be considered as limiting the scope of the features disclosedherein. For example, different types of teleoperated systems havingslave devices at worksites can make use of actuated controlled featuresdescribed herein.

FIG. 2 is a perspective view of an example of a hand controller 200which can include one or more features described herein, according tosome implementations. In some examples, hand controller 200 can be anungrounded master controller configured to be held by a user's hands andthat is mechanically ungrounded during its operation. For example, handcontroller 200 can be used as a master control device 122 as describedwith reference to FIG. 1, or in other master control applications. Handcontroller 200 is contacted and held by a user to provide controlsignals to one or more systems in communication with the handcontroller. In this example, hand controller 200 includes a control body201 and two grip portions 204, labelled as 204A and 204B.

Control body 201 includes a central portion 202 and an extension member220. Central portion 202 is an elongated member as shown and has acentral longitudinal axis 203 along which the central portion extends.The control body 201 can be moved in space and the position and/ororientation of the control body 202 (or another portion of the handcontroller 200) in space can be sensed.

One or more sensors can detect, and/or can enable the detection of, theposition and orientation of the control body 201 in space, e.g., in aworking environment or workspace of the hand controller 200. Inimplementations where the controller 200 is mechanically ungrounded, thecontrol body 201 is effectively unconstrained for both position andorientation motions within the user's reachable workspace and a sensingworkspace. Some examples of sensing systems able to sense the positionand orientation of the control body 201 are described above. In someimplementations, the control body 201 can include a component that canbe tracked by a sensing system that is located externally to the handcontroller 200, e.g., one or more magnets, electromagnetic signalemitters, optical patterns, etc. In some implementations, the controlbody 201 can include a receiving component that receives signals emittedby an external system to assist in determining position and/ororientation of the control body 201 in space. In some implementations,the control body 201 can include one or more sensors or sensorcomponents operative to sense and/or assist an external sensor indetecting position and orientation of the control body 201.

For example, a sensor can track position and/or orientation of the handcontroller 200 in a working environment relative to a fixed referencepoint. In some examples, a sensor Cartesian coordinate system (Xs, Ys,Zs) may be generally centered at the sensor. The sensor may serve totrack movements, such as the movements of the hand controller 200 and/oruser's wrist and forearm, to control a slave device, e.g., rotate and/ortranslate a surgical tool end effector or other instrument. In someapplications, the reference coordinate system may be a finger gripcoordinate system, such that any movements measured in the sensorcoordinate system may be transformed by an applied transformation fromthe sensor coordinate system to the finger grip coordinate system. Insome examples, motion sensors (accelerometers, gyroscopes, etc.) can beused and provided within the control body 201.

In some implementations, the hand controller 200 does not include asensor or sensor component for tracking its position and orientation inthe workspace, and an external sensor system can perform such tracking(e.g., one or more cameras capturing video and/or motion occurring inthe workspace, and a control system detecting and tracking the handcontroller in the workspace by examining the captured video or recordedsensor data, etc.).

In some examples, the position, orientation, and/or motion of the handcontroller 200 in three-dimensional space can be sensed to controloperation of a teleoperated slave device. For example, position,orientation, and motion of the hand controller with respect to areference position in three-dimensional space can be used to control acorresponding position, orientation, and motion of an arm assemblyand/or end effector instrument of a slave device in its availableworkspace and degrees of freedom.

Grip portions 204A and 204B are coupled to the central portion 202 ofthe control body 201 (generally referred to as 204). In someimplementations, a single grip portion 204 can be provided on the handcontroller 200, or more than two grip portions 204 can be provided.

Each grip portion 204A and 204B can each include a grip 206 which is aposition or member at which to contact a user's finger. Each grip 206can have a surface that is shaped to receive a finger pad of the user.In various example implementations, the grip 206 has a contact surfacethat is flat (e.g., parallel to the grip member portion that extendsfrom the central portion 202), concave (curved inward to form a valleyto fit the finger), or convex (curved outward to form a bump or shellengaged by the finger) to provide engagement and secure contact with thefingers of the operating hand. In one example, a convex surface can besuitable in some implementations to make fingertip control of the grips206 easier, e.g., where the fingertips can roll across the convexsurface. The grips 206 can have a tapered surface in some examples. Forexample, tapered grips can taper inwards, such that the grips are at anangle to the surface of the grip portions to which they are coupled.Some examples of concave and convex grips are described below withrespect to FIGS. 11 and 12.

Some implementations can provide protrusions that extend outwardly fromthe grip 206 in which to cradle a finger, or an aperture in which afinger is inserted. Some implementations of a grip 206 can includetexturing such as bumps, ridges, or other patterns of features (someexamples described below) to engage the user's finger. A multiple-fingergrip 206 can be used in some implementations, where multiple fingersengage a single grip 206. For example, a grip 206 can include adjacentconcave depressions (or protrusions forming adjacent spaces that cradlefingers) to engage two or three fingers side-by-side, e.g., the secondand third fingers, the third and fourth fingers, or the second, third,and fourth fingers (e.g., with the other grip 206 engaging the thumb).

In some examples, as shown, each grip can include a finger loop 208which can be used to hold a finger to the associated grip 206. In someexamples, a finger loop can include a fastener (e.g., hook and loopfasteners, buckle, etc.) to allow tightening of the loop around thefinger. In some implementations, one or more buttons or other controlscan be provided on or coupled near to the finger loops 208 (e.g.,similar to buttons 1128 of FIG. 12). The two grip members 306 arepositioned on opposite sides of a central portion 303 of the handle 302,where the grip members 306 can be grasped, held, or otherwise contactedby a user's fingers. The two finger loops 304 are attached to gripmembers 306 and can be used to secure a user's fingers to the associatedgrip members 306. The user may also contact other portions of handle 302while grasping the grip members 306.

Each grip portion 204A and 204B can also include an associated gripmember 210 that is pivotally or rotatably attached to the control body202 of the hand controller 200 at a pivoting end of the grip member. Agrip 206 and finger loop 208 is coupled to a finger end of eachassociated grip member 206. The grip portion can be moved, e.g., by auser, in an associated degree of freedom 212 with respect to the controlbody 202, where the finger end of the grip member 210 is moved. Thus theassociated grip 206 and finger loop 208 coupled to that finger end aremoved with the grip member 210. For example, the grip members 210 can bemoved simultaneously in a pincher-type of movement (e.g., toward or awayfrom each other). In some implementations, one of the grip members 210can be moved in its degree of freedom 212 while the other grip member210 (or other grip 206) can be fixed with reference to the control body202. In some implementations, both grips 206 can be fixed with referenceto the control body 202, e.g., grips 206 can be coupled directly to thecontrol body 202 or can be coupled to a structure that does not movewith respect to the control body 202.

One or more sensors (not shown) coupled to the hand controller 200 candetect the positions of the grip members 210 in their degrees of freedom212 and send signals describing the positions to one or more controlcircuits of the system to which the controller 200 is connected, e.g.,teleoperated surgical system 100 or other system. For example, opticalencoders, potentiometers, or other sensors can be used. In someexamples, the control circuits provide control signals to theteleoperated slave device 102, an example of which is described withreference to FIG. 21. For example, the positions of the grip members 210in degrees of freedom 212 can be used to control jaws or othercomponents, or any of various degrees of freedom, of an end effector ofa controlled slave device (e.g., teleoperated slave device 102). In someexamples, the two finger grips 206 of hand controller 200 can be movedtogether or apart by the user in pincher motions to, for example,correspondingly move forceps, pliers, or other instrument end effectorof the teleoperated slave device.

In some implementations, one or more springs or other actuators can beprovided between each grip member 210 and the control body 202, toprovide a resistive force in particular directions of the grips 206(e.g., movement in directions toward each other in degree of freedom212.) In some implementations, the actuators can provide a restoringforce to the grip member 210 toward an open position of the grip member.When the user reduces finger force on the associated grip 206, the gripmember 210 may be moved toward the open position by the restoring force.In various implementations, the resistance and/or restoring force on thegrip members can be provided by various types of actuators, e.g.,passive actuators that provide a passive resistive force to movement(such as springs that provide an increasing resistive force the closerthe grip member is moved to the central portion, or dampers, resistiveelements, etc.) and/or active actuators (motors, voice coils, etc.) thatprovide an active force. In some implementations, the actuator(s)provide forces that are varied based on a control signal provided to theactuator(s) from a controller. In some examples, the grip members caninclude a power assist mechanism using one or more actuators to provideassistive force to the grip members and assist the user when moving agrip member between positions. In some implementations, other types offorces can be provided on the grip portions 204, e.g., damping force,force pulses or vibrations, etc. In some examples, a sensor and/oractuator can be housed in control body 202 which is coupled to the gripmembers 210 by a transmission.

The control body 201 of the hand controller 200 also includes anextension member 220 that forms a proximal end 222 of the control body201. In some implementations, the extension member 220 and the centralportion 202 form a unitary control body 201. In some implementations,extension member 220 is coupled to a separate central portion 202. Insome implementations, the extension member 220 is removable from thecentral portion 202 and, for example, can be replaced at the proximalend 222 of the control body 201 by a differently-sized and/ordifferently-shaped extension member.

In some implementations, at least a portion of the extension member 220(e.g., the proximal end of the control body) has an outer surface thatis spaced further from the longitudinal axis 203 than the outer surfacesof the central portion 202, e.g., the portion of the extension member220 has a greater radius at its cross-section than the central portion202. This allows the extension member to be easily contacted or graspedby the user's hand. In the shown implementation, the extension memberincludes at least a portion of a spherical surface, e.g., ahemispherical surface or a similarly-curved surface at the furthestproximal end of the extension member 220. Other forms, shapes, andfeatures of the extension member can be provided in otherimplementations. For example, the extension member can have acylindrical or oblong shape, rectangular or other polygonal faces,rounded corners, etc.

In this example implementation, the extension member 220 (e.g., proximalend of the control body 201) is axisymmetric, e.g., has an symmetricshape with respect to revolution about a longitudinal axis of thecontrol body. In some implementations, the extension member 220 has ashape that is asymmetric with respect to the longitudinal axis 203 ofthe control body 201, some examples of which are described below. Someimplementations can provide a rigid extension member 220 that is notdeformable by the user's hand, and other implementations can provide adeformable, compressible, or flexible extension member 220 or adeformable, compressible, or flexible covering to the extension member200, e.g., made of rubber, foam rubber, neoprene sponge, or otherdeformable material. Such materials can be used in any of theimplementations described herein.

In some implementations, one or more physical connections, e.g., atether connection such as a cable, may extend out of the extensionmember 220 to connect the hand controller to a master control system,some examples of which are described below.

During usage of the hand controller 200, the proximal end 222 of thecontrol body 201, e.g., the extension member 220, is typicallypositioned closer to the palm of the user than the grips 206. Thecontrol body 201 has a length and/or shape that is configured toselectively engage or contact (ground) the proximal end of the controlbody in or against the palm of the hand of the user while the handengages (e.g., pinches or holds) the thumb grip portion 204 (e.g., grip206) with a thumb of the hand and the finger grip portion 204 (e.g.,grip 206) with a finger of the hand. In some implementations, theextension member 220 is made sufficiently small to allow this selectiveengagement. For example, during use of the hand controller 200, at leasta portion of the curved surface of the extension member 220 can beselectively made to engage (e.g., contact) the user's palm bymanipulating the control body 202 and grip members 210 using theportions (e.g., tips) of the user's fingers contacting the grips 206.Engagement with the palm in this context refers to contact between theextension member and the palm, e.g., grounding of the control body tothe hand of the user. Disengagement from the palm refers to removingsuch contact between control body and palm.

Thus, the control body is configured to allow the proximal end 222(e.g., extension member 220) to be selectively engaged by the palm ofthe hand and selectively disengaged from the palm of the hand. Forexample, the control body is configured to allow the proximal end to bemoved by manipulation of the thumb and the grip finger of the hand onthe control body to move the proximal end into selective engagement withthe palm. In some implementations, the control body is configured toallow the proximal end to be selectively moved into engagement with thepalm using additional fingers of the hand that are different than thethumb and the finger contacting the grips, e.g., the third, fourth,and/or fifth fingers, as described in greater detail below.

In some examples, the extension member 220 is selectively and naturallycradled or surrounded by the user's palm, without engagement or contactwith the palm, while the control body 202 and grip members 210 aremanipulated using the user's fingers, such that the extension member 220can be contacted by the palm of the user when desired or if anunintentional slippage or displacement of the controller 200 occurs inthe hand. In this example, the proximal end or extension member providessecurity because it is in a position to be grasped if additionalsecurity is desired or needed. The extension member 220 is naturallycradled near to the hand due to the natural curvature of the additional,non-gripping fingers (e.g., fourth and fifth fingers), so that if thecontroller slips, the proximal end will contact at least one of thepalm, fourth finger, and fifth finger, thus providing added security.For example, at any time, one or more of the non-gripping fingers cancontact the extension portion 220 to move the extension portion 220 intoengagement with the palm for added security in a grasping motion. Thesefingers can release their contact with the extension portion 220 toallow disengagement of extension portion 200 and the palm when the userdesires to have easier fingertip control of the controller 200.

In addition, the extension member 220 can be pivoted or otherwise movedout from the space adjacent to or contacting the user's palm to allow agreater range of motion of the hand controller when the user changesorientation of the hand controller in space, e.g., using the hand'sfingertips. For example, the fingers on grips 206 can act as a fulcrumaround which the controller 200 is moved. Thus, the finger tip range ofmotion of the hand controller is increased by configuring the extensionmember 220 to allow such motion. For example, the control body 201 isconfigured to allow the proximal end of the control body 201 (e.g., theextension member 220) to be moved between the thumb of the hand and thefirst (index) finger of the hand while the hand engages (e.g., pinchesor holds) the thumb grip portion (e.g., grip 206) with the thumb of thehand and the finger grip portion (e.g., grip 206) with a finger of thehand. Alternately, the proximal end (extension member 220) can be movedin a direction opposite to the area between the thumb of the hand and anindex finger of the hand. Such movement or positioning examples aredescribed in greater detail below with respect to FIGS. 6 and 7. Suchfingertip motion can be easier when the control body 201 is provided ata length such that the extension member 220 does not contact the palm ofthe hand (e.g., is positioned with space or gap between its surface andthe palm) while the user is holding the hand controller in a centeredposition, e.g., a position where the axis 203 approximately passesinto/through a center portion of the palm such as the position shown inFIG. 3.

The spherical or curved surface of the extension member 220 allows auser's palm to comfortably contact or manipulate the extension member220 when desired. This can allow the user to use an increased portion oftheir hand to manipulate the controller, e.g., in addition to thefingertips that contact the grips 206. This may be more comfortable forusers who are more familiar with using larger portions of their hands tomanipulate controllers. Furthermore, the extension member 220 canprovide increased security against dropping and/or slipping of the handcontroller 200 during use by providing a surface that is convenient tocontact or grasp by the hand. In addition, the extension member 220allows better management of holding the hand controller 200 by providinga portion to hold if the user wishes to readjust his or her fingers onthe grips 206, e.g., causing fewer inadvertent drops of the handcontroller. Furthermore, the extension member 220 at the proximal end ofthe controller can set the center of gravity of the hand controller 200further toward the proximal end 22 of the control body 201. For example,this may adjust the center of gravity closer to the grips 206 for morebalanced usage, e.g., if other weight exists at the distal end of thehand controller 200.

The ease of moving the hand controller 220 with respect to the operatinghand can also be advantageous in implementations providing detectedgestures. For example, some implementations can detect differentorientations, positions, and/or motions of the hand controller 200 inspace as gestures that can be used to activate or command variousfunctions in a teleoperated system (e.g., gesture poses and/or gesturetrajectories). For example, a first gesture pose can be to align thecontrol body 202 vertically in space pointing the distal end in onedirection, which commands a first function of the system, and a secondgesture pose can be to align the control body 202 vertically andpointing in the opposite direction, which commands a second function ofthe system. Such manipulation can be provided using the hand controller200 while still retaining the security of grasping the hand controller200 so that it is not inadvertently dropped and/or does not partiallyslip out of the hand.

In some implementations, the extension member 220 and central portion202 are rotatable about the longitudinal axis 203 of the control body201 with respect to each other. In some examples, the extension member220 is rotatable about the axis 203 independently of the central portion202 and grip portions 204A and 204B. In some examples, a rotary couplingis provided between central portion 202 and extension member 220. Thisrotation can allow the central portion 202 and grip portions 204 to berotated about the axis 203 with respect to the extension member 220,e.g., by the user's fingers, while the extension member 220 is heldengaged to the palm or other portion of the user's hand (e.g., groundedto the hand). Similarly, the extension member 220 can be rotated aboutthe axis 203 with respect to the central portion 202. In someimplementations, the extension member 220 can pivot in all axes withrespect to the central portion 202, e.g., using a spherical joint orcoupling provided between central portion 202 and extension member 220.Some examples of such an implementation are described below with respectto FIG. 15. In some implementations, the rotary position of theextension member about axis 203, and/or the rotation or pivoting motionof the extension member 220 about axis 203, can be sensed using one ormore sensors coupled to the central portion 202 and/or extension member.For example, such position or rotation can be used to sense a rotaryposition of an input control or asymmetrical feature on the extensionmember, sense a current location of a tethered connection that extendsfrom the extension member 220 (e.g., as in FIG. 5), etc. For example,sensing particular rotary positions or motions can cause the system tooutput a displayed, audio, and/or haptic indicator or alarm, exitcontrolling mode, etc. if the current position or motion matches (withina threshold amount) any of the particular positions or motions.

In some implementations, the extension member 220 can be expanded orcollapsed in its length or size to allow adjustment in controller sizeand customization to a particular user's hand. For example, theextension member 220 can have an expandable surface to allow it to beincreased in diameter, or can be configured to extend/collapse in aparticular direction or along a particular axis (e.g., along an axisparallel to axis 203). In some implementations, the extension member 220can be removed, e.g., disconnected from central portion 202, e.g., ifonly fingertip control of the controller 200 is to be used. In someimplementations, a differently-sized extension member 220 can beconnected to the central portion 202 in place of the removed extensionmember.

In some implementations, the extension member 220 can be moved orextended linearly, e.g., can be translatable along the longitudinal axis203 independently of the central portion 202 and the grip portions 204.For example, the extension member 220 can be adjusted along the axis 203to fit a particular-sized hand of the user, e.g., such that the grips206 are more comfortable to use by the fingers of the user while theextension member 220 rests against or is positioned near the user'spalm. For example, the central portion 202 can include a telescopingportion to allow the extension member 220 to be adjusted parallel toaxis 203. In some implementations, detents or other mechanisms can beprovided to lock or bias the extension member 220 at particularpositions along the axis 203, e.g., such that additional force isrequired from the user's hand to move the extension member 220 out ofthese positions.

In some implementations, a sensor can be provided in hand controller 220to sense the position and/or linear motion of the extension member 220parallel to the axis 203 and relative to the central portion 202. Forexample, a linear sensor, mechanical switch, optical encoder, opticalsensor, or other type of sensor can be used as the sensor. This sensingcan allow a controller (e.g., the control system 150) to determine thedistance or amount that the extension member 220 has been moved in aparticular direction, and/or to determine a current position of theextension member 220 in its linear range of motion. In someimplementations, such a sensor can sense a control signal activation bydetecting a threshold amount of translation of the extension member 220along the longitudinal axis 203. For example, such a sensor can be aswitch that is activated in response to the user pressing on theproximal end of the extension member 220 to move it a threshold distancetoward the distal end of the hand controller 200. In someimplementations, this activation is considered an activation of an inputcontrol with similar effect to, e.g., the activation of switch 224described below.

In some implementations, hand controller 200 can also include one ormore input controls (also referred to as an “activation control,”“activation control switch,” or “activation control button”). An inputcontrol includes one or more sensors (e.g., mechanical switches, opticalsensors, magnetic sensors, capacitive sensors, pressure sensors, etc.)that detects user input, e.g., the engagement or activation of a user'sfinger with the input control. Input controls can be used to detectactivations of control signals by the user by, e.g., detecting aposition of a finger or a threshold amount of contact with a finger ofthe user's hand. In some examples, an input control is a physicalpushbutton or sliding switch that is operative to be activated by userinput, e.g., engaged, slid, or pressed downward by at least a portion ofa finger of the user that is operating the hand controller 200. Variousother types of input controls can be also or alternatively be used toenable user activation of a control signal, e.g., optical sensor areas,capacitive sensor areas, pressure sensors, etc. The activation of aninput control causes a control signal to be output by the input control,e.g., to a control system. The control system can be in the housing ofthe hand controller 200 and/or in a separate device in communicationwith the hand controller (e.g., as described for control system 150 ofFIG. 1). In some examples, the control signal can cause activation of aparticular function provided by a system in communication with the inputcontrol as described above.

Input controls can be provided at any surface or portions of handcontrollers described herein, e.g., on the central portion 202, one ormore grip portions 204, extension member 206 (e.g., at the proximal endof the hand controller or on a different portion of the surface of theextension member), etc. An input control can be used to sense user inputto cause activation of particular system functions, and/or to sensecontact or use of the hand controller by a user.

In the example of FIG. 2, the input controls include an extension memberswitch 224 provided between the extension member 220 and the centralportion 202. Switch 224 includes a ring 226 that can be centered on thecentral axis 203 of the hand controller. Ring 226 activates (e.g.,closes or opens) the switch if the ring is linearly translated by theuser with respect to the extension member 220 and central portion 202along the central (longitudinal) axis 206. For example, pressing ring226 to move in a direction 228 toward the proximal end 222 of the handcontroller 200 can cause switch 224 to close, which causes a controlsignal to be sent to the control system to trigger an associatedfunction of, e.g., the teleoperated system 100 (e.g., camera modeactivation, a clutch control to enter or exit controlling mode, etc.).In some implementations, the switch can be an input control that outputsa variable control signal that is based on a current position of thering in a linear degree of freedom, where the current position is one ofmultiple different available positions of the ring, e.g., more than twopositions. The extension member 224 is provided in a position allowingeasy access to the switch by a variety of fingers of the hand operatingthe controller 200, and since the ring of the switch extends fullyaround the controller (around axis 203), switch 224 is accessible on anyside of the controller 200 regardless of the particular orientation ofthe hand controller in space with respect to the user's fingers.

In additional examples of input controls, a finger switch 230 can beprovided on the central portion 202 to enable control of one or morefunctions of the teleoperated system. The finger switch 230 can be asliding switch as shown (e.g., translatable parallel to the axis 203),or can be a press switch or button, optical sensor area, or other formof switch activatable by a finger. The finger switch 230 can engage auser's finger during operation of the hand controller 200. For example,the finger switch 230 can be engaged by a user's finger that is locatedbetween two fingers that operate the finger grips 206. In some examples,a thumb operates one grip 206, a grip finger operates the other grip 206(e.g., a third and/or fourth finger), and a second (e.g., index) fingerbetween the thumb and grip finger operates the switch 230.

Other switches can be positioned at other portions of the handcontroller 200, some examples of which are described below. In someimplementations, various types of controls can be provided on the handcontroller to provide input signals based on physical manipulation ofthe controls by the user's hand, such as dials, knobs, buttons, sliders,trackpads or capacitive sensors, joysticks, trackballs, pivotingswitches, etc.

In some implementations, the hand controller 200 can include one or morepresence sensors that detect that a user's hand is engaged with and/oroperating the hand controller 200. For example, optical sensors,pressure sensors, etc. can be used, e.g., at one or more of the grips204, at the extension member 220, and/or at other areas of thecontroller 200. For example, the presence sensors can be used todetermine whether a user is proximate to and/or operating the handcontroller. In some implementations, one or more input controls can beused as presence sensors.

In further example implementations, multiple presence sensors can beprovided at various locations of the hand controllers described hereinto measure user proximity at multiple locations, e.g., to avoid falsepositive and/or false negative measurement of particular hand gripconfigurations of the controller. For example, output signals from asensor that measures user presence by, e.g., sensing user proximity(such as distance) of the proximal end (e.g., extension member 220) ofthe controller to the palm can be examined by the system in combinationwith output signals from a second sensor (and/or third sensor, fourthsensor, etc.) simultaneously measuring user proximity of differentportion(s) of the controller to different portion(s) of the user's hand.For example, a second presence sensor can be located at the centralportion 202 (e.g., on one side of switch 230 toward the proximal end)and can sense proximity of a central portion 202 of the hand controllerto a segment of the index finger or thumb of the user's hand. In anotherexample, a second (or third, etc.) presence sensor can be located on oneor more grips 206 or grip members to sense user proximity of tips orother segments of fingers to the sensor. In some implementations, aparticular combination of sensing by a plurality of presence sensors ofthe hand controller that indicates hand engagement of different portionsof the users hand to particular portions of the hand controller can berequired to activate use of the controller and/or to activate particularcontrolled functions of the system, e.g., to send a control signal thatcauses entering a controlling mode or activates other functions. Suchcombination of sensing can indicate that the user's hand is in aparticular grasping configuration with respect to the hand controller.This sensing can provide a higher confidence estimate to the system thatthe user's hand is in a predetermined or intended grasping configurationwhen engaged with the hand controller, where portions of the hand arelocated in predetermined locations with respect to particular portionsof the hand controller.

In further examples of presence sensing, one or more input controls canbe used as presence sensors to sense presence of the hand with respectto the hand controller and can be used by the system to estimate auser's intent with respect to activating one or more functions of theteleoperated system, e.g., to enter a controlling mode, exit acontrolling mode, activate a function of a slave device, etc. Forexample, a confirmation input control can be provided on the handcontroller 200 that, when activated by the user, indicates that the useris ready to activate one or more functions (e.g., enter a controllingmode, activate one or more other particular functions, etc.), and whichenables such activation to be performed (e.g., in response to receivinginput from one or more other input controls). In some examples, thisactivation is not enabled prior to activation of the confirmation inputcontrol. In some implementations, this activation can be enabled for aparticular time period after activation of the confirmation inputcontrol. The confirmation input control can be an additional, dedicatedinput control used for such indication, or can be an input control thatis also used for other functions as described herein. In some examples,the confirmation input control can be a button, switch, sensor (e.g.,optical or capacitive sensor), or other control on the hand controllerthat is activated by squeezing the proximal end (e.g., extension member220) with the palm, or activated with fingers that are not used toactuate a gripper mechanism (e.g., finger(s) that can activate anextension member switch 224, extension buttons 1010, etc., of theproximal end).

In further examples of user presence sensing, a determination that userpresence is no longer sensed by one or more presence sensors can be usedto activate particular functions. For example, determining that a userhas disengaged a hand from the hand controller, as sensed by aparticular presence sensor, can cause the system to enter amore-persistent type of non-controlling mode in which explicitcommand(s) are required to be input by the user for the system tore-enter controlling mode. For example, an explicit command can beprovided to the control system via a different input control of the handcontroller (e.g., a switch, button, etc.), or a command provided to thecontrol system via a different input device (e.g., a foot pedal or otherfoot control, a separate sensor that senses the user or a portion of theuser such as the user's head, gaze, etc.). In some implementations,sensing user disengagement from the hand controller by a specificpresence sensor (e.g., a different presence sensor than sensor(s)associated with the more-persistent controlling mode described above)can cause the system to enter a less-persistent non-controlling mode,from which re-entering controlling mode has fewer or simpler inputsensing requirements than for the more-persistent non-controlling mode.For example, such input sensing requirements can include sensing userpresence at the same specific presence sensor after such disengagementis sensed (e.g., within a threshold time period after disengagement issensed).

Some implementations of the hand controller 200 can include a distalelement 236 coupled at the distal end 238 of the control body 202, e.g.,at the opposite end of the control body 202 to the extension member 220.The distal element 236 can, in some implementations, include one or moresensors or sensor components used for tracking the position and/ororientation of the hand controller 200 in space, e.g., in a workingenvironment such as a surgical environment. For example, receivers,transmitters, motion sensors, and/or other sensor components can beprovided in the distal element 236.

Alternatively, these sensor(s) can be positioned at various locations onor in the housing of the hand controller 200, e.g., on the centralportion 202 below or above the grip portions 204. In someimplementations, the hand controller 200 can include one or more sensorsor sensor components operative to sense and/or assist an external sensorin detecting position and orientation of the hand controller 200. Forexample, motion sensors (accelerometers, gyroscopes, etc.) can be usedwithin the hand controller 200 in some implementations. In variousimplementations, the sensor may be a six degree of freedom (6 DOF)electromagnetic (EM) sensor, an optical tracking sensor, a fiber opticshape sensor, or another type of sensor.

In some implementations, the sensor may be one component of a sensorsystem, where additional components of the sensor system are positionedexternal to the hand controller 200 (e.g., hand-tracking transceiver 130of FIG. 1). For example, the hand controller 200 can include a component(e.g., sensor 234) that can be tracked by a sensing system that islocated externally to the hand controller 200, e.g., a component such asone or more magnets, electromagnetic signal emitters, optical patterns,etc. In some implementations, the hand controller 200 can include asensor receiving component that receives signals emitted by an externalsystem to assist in determining position and/or orientation of the handcontroller 200 in space. In some implementations, the hand controller200 includes a sensor or sensor component for tracking its position andorientation in the workspace, and an external sensor component canperform such tracking (e.g., one or more cameras capturing video and/ormotion occurring in the workspace, and a control system detecting andtracking the hand controller in the workspace by examining the capturedvideo or recorded sensor data, etc.)

In some implementations, the distal end element 236 includes a weightedelement to provide a weight on distal end of the control body 202, e.g.,to provide a particular weighted balance to the hand controller 200 incoordination with the weight at the proximal end from the extensionmember 220. These weights can be selected to provide a center of gravityat a particular location or portion of the hand controller 200. In someimplementations, a distal weighted element at the distal end of thecontrol body (e.g., a sensor and/or one or more additional weightedelements) and a proximal weighted element at a proximal end of thecontrol body (e.g., extension member 220 and/or one or more additionalweighted elements) are weighted to provide a neutral balance to thecontroller 200, e.g., a balance such that the center of gravity of thecontroller 200 is in a middle portion of the controller in proximity tothe grips 206. In some examples, the center of gravity can be providedin a region of the controller that is on the control body 202 andbetween (e.g., approximately centered between) the thumb and fingergrips 206.

In some implementations, the hand controller 200 may include sensorsthat provide safeguards against the hand controller 200 inadvertentlydropping. For example, in some implementations, the thumb grip 204and/or the finger grip 214 may include a sensing mechanism (not shown)that senses contract with the fingers of the user. In someimplementations, the hand controller may include an accelerometer (notshown) that senses if the hand controller drops, e.g., to the ground.The sensing mechanism and/or the accelerometer may detect if the hand ofthe user releases the hand controller 200. For safety, the handcontroller 200 may then discontinue control of the slave devices of theteleoperated system 100.

In some implementations, one or more tethered connections can beconnected to and extend out of the distal end 238 and/or the distalelement 236. Various control signals from sensors of the hand controller200 can be output via the tether connection, and/or various signals froma control system can be received. For example, the tethered connectionscan be cables that are attached to a control system, e.g., controlsystem 150 of FIG. 1. In some implementations, the tethered connectionscan extend out of the proximal end of the hand controller, e.g., out ofthe extension member 220, instead of out of distal end 238, as describedin greater detail below. In some implementations hand controller 200 isnot tethered by physical connections, and can communicate with thecontrol system via wireless signal communications. For example, awireless transmitter can be provided in some implementations within thedistal element 236, where the transmitter is configured to send wirelesssignals to a master control system based on position, orientation,and/or motion of the control body 201 in space, based on the thumb gripand the finger grip portions 204 in their degrees of freedom, and/orbased on the activations of input controls of the hand controller.

In some implementations, the hand controller 200 can be a mechanicallygrounded controller. For example, the control body 201 can be coupled toa mechanical linkage that is coupled to the ground, providing a stableplatform for the use of the hand controller. For example, the handcontroller 200 (or other hand controller implementations herein) can becoupled to a mechanical linkage that is coupled to the ground or anobject connected to ground, providing a stable platform for the use ofthe hand controller 200. For example, the control body 201 of the handcontroller 200 can be coupled to a grounded mechanical linkage at thedistal end 212 of the hand controller. The mechanical linkage canprovide six or more degrees of freedom to the hand controller 200. Someexamples of such linkages are described below with reference to FIGS. 17and 18, and in U.S. Pat. No. 6,714,839 B2, which is incorporated hereinby reference.

FIG. 3 is a perspective view of the master hand controller 200 beingmanipulated by a user's hand, according to some implementations. In thisexample, the user's thumb 302 is engaged with grip 206A and secured byfinger loop 208A. The user's third finger 304 is engaged with grip 206B(not shown; e.g., see FIG. 4) and secured by finger loop 208B. Theuser's second finger 306 is engaged with the finger switch 230, e.g., toslide the finger switch 230 forward and/or back to different activationpositions.

In this example, the extension member 220 is engaged with the user'spalm, e.g., contacted by the palm to more reliably grip the handcontroller 200 while manipulating the grip portions 204 and fingerswitch 230 with the user's fingers.

FIG. 4 is a perspective view of the master hand controller 200 beingmanipulated by a user's hand to activate a switch adjacent to theextension member, according to some implementations. For example, FIG. 4can be a preceding or following movement or action by the user's handrelative to the hand position shown in FIG. 3. In the example of FIG. 4,the user has moved the second finger 306 to engage the extension memberswitch 224 positioned between the control body 202 and the extensionmember 220. For example, the second finger 306 can be moved toward theextension member 220 to move the ring 226 of the switch 224 toward thepalm of the user (e.g., toward the proximal end of the hand controller200) which activates the switch 224. For example, the ring 226 canreceive a restoring force provided by a spring or other actuator, whichcauses the ring 226 to return to its (unactivated) position if thesecond finger 306 removes sufficient force that was applied toward theextension member 220.

In some implementations, the ring 226 can also or alternatively beactivated using a different finger of the hand of the user. For example,the fourth finger, fifth finger, etc. can engage the ring 226 and moveit toward the proximal end of the hand controller 200, e.g., on a sideof the controller 200 nearest to those fingers.

FIG. 5 is another perspective view of the master hand controller 200being manipulated by a user's hand to activate a switch adjacent to theextension member, according to some implementations. FIG. 5 shows anexample of an underside view of the hand controller and hand positionsshown in FIG. 3, or after the user has rotated the hand controller andhand in space.

In this example, the user has used a fourth finger 502 to activate theextension member switch 224 by engaging and moving the ring 226. Forexample, the second finger 306 of the user (shown in FIG. 3) cancontinue to engage and activate the switch 230 (shown in FIG. 3) whilethe ring finger 502 engages and activates the extension member switch224. In some implementations, the fifth finger 504 can alternately beused to engage the ring 226 to activate the extension member switch 224.A second finger switch 231 can be provided on the opposite side of thecontrol portion 202 to the finger switch 230 shown in FIG. 4. In someexamples, the second finger switch 231 can be activated by finger 502 orfinger 504, e.g., a finger that is different than the finger 396 thatactivates the switch 230.

The example of FIG. 5 also shows an example of a tethered connection(e.g., electrical wire or cable) 510 extending from the extension member220 at the proximal end of the hand controller. The connection 510 canbe connected to a control system as described above and used tocommunicate electronic signals to and from the hand controller 200. Inthis example, the user can manipulate the hand controller 200 while thetethered connection 510 extends out the side of the user's hand/palmopposite to the finger side of the palm that engages the extensionmember 220. Since the extension member 220 may rest in the user's palm,in some implementations the tethered connection 510 can extend from thecontrol body or extension member at least partially radially from thecentral longitudinal axis 203, instead of extending out of the center ofthe extension member 220 along axis 203. This may allow the tetheredconnection 510 to avoid interfering with the palm. One example is shownin FIG. 7. In implementations allowing rotation of the extension member220, the point 512 where the tethered connection meets the extensionmember 220 can be rotated about the central axis 203, allowing thetethered connection to extend out of the controller 200 at a point thatis adjustable by the user about axis 203, e.g., to accommodate thecurrent grip of the user on the controller.

In some implementations, such a proximal end tether can provideincreased security due to the weight of the tether being closer to theuser's hand and palm, and may allow less collisions with the tethersince it is out of the path of motion of the hand controller 200 inspace, unlike in some implementations of a tether extending from thedistal end of the hand controller 200. In implementations in which theextension member 220 can rotate with respect to the central portion 202,this rotation allows the tethered connection 510 to maintain itsposition extending away from the palm of the user's hand while thecentral portion 202 and grips 204 are rotated in space. In someimplementations, a tethered connection can be connected and extendedfrom the distal end of the hand controller, as described above.

FIG. 6 is a perspective view of the master hand controller 200 beingmanipulated by a user's hand to a first orientation, according to someimplementations. FIG. 6 shows an example of a view of the handcontroller and hand positions after the user has moved the extensionmember 220 of the hand controller 200 out of the space adjacent to orcontacting the palm of the hand.

In this example, the user has rotated the control body 201, includingcentral portion 202 and extension member 220, using the thumb (firstfinger) 302 and third finger 306 of the hand on the grips 206. Thelengths of the central portion 202 and extension member 220 aresufficiently short to allow the extension member 220 to pivot out of thespace near the palm and in a direction through the space between thumb302 and second finger 306, e.g., past or adjacent to the webbing betweenthe thumb and second finger (forefinger) of the hand, such that theextension member 220 is no longer in the space adjacent to the palm andis no longer contacting the palm. In some examples, the length of thehand controller 200 thus allows the user to freely manipulate theorientation of the hand controller 200 in space to control functions ofa slave device (e.g., teleoperated slave device 102 of FIG. 1) withoutsevere restriction on those orientations due to interference with theuser's hand.

FIG. 7 is a perspective view of the master hand controller 200 beingmanipulated by a user's hand to a second orientation, according to someimplementations. FIG. 7 shows an example of a view of the handcontroller and hand positions after the user has moved the extensionmember 220 of the hand controller 200 out of the space adjacent to orcontacting the palm of the hand. In this example, the extension member200 has been moved in a direction different than and/or opposite to thedirection of movement of the controller 200 shown in FIG. 6, e.g., inthe direction away from the space between the first finger 302 and thesecond finger 306. In this example, the user has rotated the centralportion 202 and extension member 220 using the thumb (first finger) 302and third finger 306 of the hand on the grips 206.

The lengths of the central portion 202 and extension member 220 aresufficiently short to allow the extension member 220 to pivot out of thespace near the palm such that the extension member 220 is no longer inthe space adjacent to the palm and is no longer contacting the palm. Insome examples, the length of the hand controller 200 thus allows theuser to freely manipulate the orientation of the hand controller 200 inspace to control functions of a slave device (e.g., teleoperated slavedevice 102 of FIG. 1) without severe restriction on those orientationsdue to interference with the user's hand.

The example of FIG. 7 also shows an example of the tethered connection(e.g., electrical cable) 510 extending from the extension member 220 asdescribed with reference to FIG. 5. In this example, the user canmanipulate the hand controller 200 while the tethered connection 510extends from the extension member 220 on a side of the extension memberthat is opposite to the user's palm.

FIG. 8 is a perspective view of another implementation of a handcontroller 800. Hand controller 800 includes many of the same elementsdescribed above for controller 200 of FIG. 2 and these elements arenumbered identically to FIG. 2.

Hand controller 800 includes an asymmetrical extension member 802. Forexample, the extension member 802 has a shape that is asymmetric withrespect to the longitudinal axis 804 of the control body 201. In thisexample, the asymmetric shape includes an extended portion 806 thatextends asymmetrically to one side of and approximately perpendicularlyto the longitudinal axis 804. For example, during operation of the handcontroller 800, portion 806 can extend into and/or contact the palm ofthe user's operating hand, and portion 806 can be receptive to and/orgrasped by one or more fingers of the user, e.g., the fourth and/orfifth fingers. The asymmetrical extension member 802 can have a smoothand curved surface that allows it to move easily out of the palm of theoperating hand.

FIG. 9 is a perspective view of another implementation 900 of a handcontroller. Hand controller 900 includes many of the same elementsdescribed above for controllers 200 and 800 of FIGS. 2 and 8 and some ofthese elements are numbered identically. Hand controller 900 includes anasymmetrical extension member 902 rotatably coupled to a central portion202 of the hand controller 900. For example, the extension member 902has an asymmetric shape with respect to the longitudinal axis 904 of thecentral portion 202 a. In this example, the asymmetric shape includes ahandle portion 906 that extends to one side of and perpendicularly tothe axis 904, e.g., a portion of the proximal end extendingasymmetrically to one side of the longitudinal axis of the control body.The asymmetrical extension member 902, including handle portion 906, canhave a smooth and curved palm-facing surface 905 that allows it to moveeasily in and out of the palm of the operating hand, e.g., especially inthe pitch-angle directions of arrow 907.

The finger grip members 204 can be configured to be receptive to a firstfinger, second finger, and/or third finger of the hand, and the proximalend, e.g., extension member 902, can be configured to be receptive to athird finger, fourth finger, and/or fifth finger of the hand. Forexample, during operation of the hand controller 900, the handle portion906 can be grasped by one or more fingers of the user, e.g., in variousconfigurations: the third, fourth, and fifth fingers; the fourth andfifth fingers; the third and fourth fingers; the third and fifthfingers; the third finger; the fourth finger; or the fifth finger. Insome implementations, as shown, the handle portion 906 can includeindentations 908 on a side of the handle portion 906 that is, e.g.,opposite to the surface 905 contacted by the palm or other handsurfaces. The indentations 908 are spaced to be receptive to and engageone or more fingers of the user's hand operating the hand controller900, e.g., the fourth and fifth fingers in some examples. In someimplementations, indentations, protrusions, or other features can bespaced at various locations of the handle portion 906 in otherconfigurations.

FIG. 10 is a perspective view of another example implementation of anextension member 1000. Extension member 1000 is an asymmetricalextension member similar to the extension member 902 shown in FIG. 9,including indentations 1008 in handle portion 1006 similar to theindentations 908. Additionally, extension member 1000 includes one ormore input controls coupled thereto, which are configured to detect athreshold amount of contact with a finger of the hand. In this example,each input control is a button 1010A and 1010B (generally referred to asbutton 1010) provided in an associated indentation 1008 of the handleportion 1006. For example, one of the user's fingers can engage anindentation 1008 and activate (e.g., press) the button 1010 provided inthat indentation. In some examples, the fourth finger of the operatinghand can engage button 1010A, and the fifth finger can engage button1010B. In some implementations, each button 1010 can provide a differentcontrol signal when activated by the user. Any number of buttons 1010 orother types of switches or controls can be provided on the extensionmember 1000 to enable user activation of various control signals. Forexample, a larger button or activation switch can be provided on thegrip portion 1006 that can be squeezed or pressed by the user's fingersand/or palm. In some implementations, an input control can be providedon the side(s) of the extension member 1000 and/or handle portion 1006and can be activated by a finger of the hand, e.g., the first and thirdfingers.

FIG. 11 is a top plan view and FIG. 12 is a perspective view of anotherexample implementation of a hand controller 1100 that can be usedsimilarly to hand controller 200. Controller 1100 includes a centralportion 1102 and grip portions 1104, including grip portion 1104A andgrip portion 1104B, generally referred to as 1104. In this example, thegrip portions 1104 are provided as pieces that are connected to thecentral portion 1102 at a location 1111 (or at a location 1110 in someimplementations). For example, the control body 1102 can include aprotrusion 1112 that extends through an aperture 1114 in the gripportions 1104 at the location 1110. In some implementations, the gripportions 1104 can be formed as a single piece or unitary grip member inwhich the grip portions are joined, where the grip portions 1104 canmove relative to each other based on flex in the unitary grip member,e.g., at or near location 1110.

In this implementation, grip portions 1104 include respective grips 1106and grip members 1108. For example, grips 1106 can be end portions ofthe grip portions 1104 that have features configured to receive andengage a user's fingers. In this example, indentations are provided asgrips 1106 near the distal ends of the grip members 1108, where theindentations are shaped to fit the touching side (pads) of a user'sfingers.

Furthermore, in some implementations, the surface of the grip portions1104 can be textured to engage the user's fingers with more friction andgrip than a smoother surface. For example, a grid or other pattern ofsmall, raised circular bumps are provided over the outer surfaces ofgrips portions 1104, e.g., opposite to the central portion 1102, asshown in FIGS. 11 and 12. Other sizes, shapes, and patterns of raisedbumps or other texture features can be used in various implementations(e.g., long ridges, random or irregular patterns, etc.).

In some implementations, the two grip portions 1104 are not connected toeach other at location 1110 and are configured to move with respect toeach other about a pivot point at location 1111 when pinched andreleased by fingers of the user. Grip portions 1104A and 1104B can pivotrelative to central portion 1102 as indicated by arrows 1107 in FIG. 11.For example, the grip portions 1104 can be moved toward each other usinga pinch motion with fingers of the user engaging the grips 1106. In someimplementations, a spring 1126 (or actuator) can be coupled between gripmember 1104A and central portion body 1102, and/or a spring 1126 (oractuator) can be coupled between grip member 1104B and central portion1102. Springs 1126 provide a restoring or spring force on the gripmembers 1104, which biases the grip members 1104 toward a neutralposition shown in FIGS. 11 and 12. In some implementations, a unitarygrip portion that includes both grip portions 1104A and 1104B can be aflexure, e.g., providing flex at or near location 1110. Such a unitaryflexure can provide a restoring force on the grip members 1104 inaddition to or instead of a restoring force provided by springs 1126.

Hand controller 1100 includes multiple input controls in the example ofFIG. 11. For example, a button 1120 is positioned on the top side at thedistal end of the central portion 1102. This button can be pressed tosend a control signal to the control system to activate an associatedfunction of the connected system, e.g., teleoperated surgical system 100of FIG. 1. A slider switch 1122 is similar to the finger switch 230described with respect to FIG. 2. A control wheel 1124 is positioned, inthis example, on the control body 1102 closer to the proximal end of thecontrol body, but can be positioned in other locations of the handcontroller. The control wheel 1124 can be rotated by a user's finger toprovide a user-adjusted control signal based on the position or motionof the control wheel 1124 that can be used to cause adjustment ofparameters or functions of the teleoperated system 100, scroll displayedinformation on a display screen, etc.

Input controls can also be positioned on either or both of the gripmembers 1104. For example, two buttons 1128 are shown positioned on thetop surface of grip member 1104A. In some examples, these buttons can bereached and activated by a user's finger (e.g., thumb) that presses onthe grip 1106 of the grip member 1104A. This thumb activation canprovide a more stable reaction force on the hand controller 1100compared to using a different finger to activate the buttons.Alternatively, a different finger can select the buttons. A variety ofother types of controls can be positioned in these and other areas ofthe hand controller 1100, e.g., trackballs, joysticks, dials, etc.

In some examples, particular control functions of a teleoperated slavedevice can be mapped to the activation of the finger controls of handcontroller 1100. Such control functions can be re-mapped to otherfunctions in some implementations. Such functions can include, forexample, a swap function for button 1130 allowing control of a firsttelemanipulator arm or instrument to be swapped to a second arm orinstrument; a camera function and/or clutch function for slider switch1122 (e.g., one function for one switch position, the other function forthe other switch position); a user interface scroll function for thecontrol wheel 112, allowing scrolling of displayed interface elements;and energy output for slave instruments mapped to the buttons 1128.

An extension member can be coupled to the hand controller 1100 in someimplementations. In some examples, the extension member can be coupledat the protrusion 1112 at the proximal end of the hand controller 1100.For example, the extension member can be any of the implementations ofextension members described with reference to FIGS. 2-10. Furtherexamples of an extension member that can be used with hand controller1100 or other hand controller implementations herein are describedbelow.

FIG. 13 is a perspective view of another example of a hand controller1300 that can be used similarly to hand controller 200 and includes anextension member having one or more finger apertures (e.g., fingerrings). In this example, a portion of the hand controller 1300 thatincludes the central portion 1302 and grip members 1304 is similar tothe corresponding portion including components 1102 and 1104 in the handcontroller 1100 of FIG. 11. In some implementations, other componentscan be used for the central portion 1302 and grip members 1304, e.g.,similarly as in other implementations described above.

A control body 1301 includes the central portion 1302 and extensionmember 1306. Extension member 1306 can be coupled to the central portion1302 and/or grip members 1304. In some examples, extension member 1306can be rotatably coupled to a protrusion 1308 at the proximal end of thecentral portion 1302 and/or grip members 1304, which can be a protrusionof the central portion 1302 through a unitary piece that includes gripmembers 1304 similarly as described above for hand controller 1100 ofFIG. 11. In some implementations, protrusion 1308 can be included aspart of the unitary member of grip members 1304 or other component ofthe hand controller 1300. The extension member 1306 can be coupled tothe central portion 1302 and/or grip members 1304 by a rotary coupling,such that extension member 1306 can be rotated about axis 1310 withrespect to the central portion 1302 and grip members 1304. In someimplementations, the extension member 1306 can be rigidly coupled to thecentral portion 1302 and/or grip members 1304.

In some implementations, the extension member 1306 can be coupled to thecentral portion 1302 and/or grip members 1304 by a rotary coupling thatincludes two magnets. The magnets on the coupled portions can beattached to each other using magnetic force. Other types of couplingscan be used in other implementations. In some examples, the coupling canbe spherical as described below with respect to FIG. 15. In anotherexample, the coupling can include two rotary couplings and a linkbetween and connecting these rotary couplings, e.g., similar to auniversal joint or a gimbal. In another example, the coupling can allowtranslation of the extension member 1306 with respect to the centralportion 1302 and grip members 1304, e.g., in-out movement, such as anelongated coupling member inserted within a receiving coupling memberlike a sheath. In another example, the coupling can include a flexibleconnecting portion, e.g., like a trunk or rope, allowing the extensionmember motion with respect to the central portion 1302 as the flexibleconnecting portion bends. In further examples, the coupling can bedetachable to allow the extension portion 1306 to be attached andremoved from the central portion 1302 and/or grip portions 1304. Forexample, attracting magnets can be located on the extension member 1306and the protrusion 1508.

The finger grip members 1304 can be configured to be receptive to afirst finger, second finger, or third finger of the hand, and theproximal end, e.g., extension member 1306, can be configured to bereceptive to a fourth finger and/or fifth finger of the hand. In thisexample, extension member 1306 includes one or more finger apertures1312 which can receive fingers of the user, and in some implementationsare formed by finger rings 1314 in or coupled to the extension member1306. In this example, the finger apertures 1312 are positioned in theextension member 1306 to receive the last two fingers of the user's handwhile the first two fingers and thumb of the hand manipulate the gripmembers 1304 and central portion 1302. For example, such apertures 1312can cause the hand controller 1300 to be engaged by the hand of the userand allow the user's hand to grasp the hand controller 1300 moreeffectively, e.g., by reducing the possibility of inadvertent releases,slips, or drops of the hand controller 1300 by the user. In thisexample, each aperture 1312 can receive one finger of the user's hand.In some implementations, each of one or more of the finger apertures1312 can be made large enough to receive two or more fingers of thehand.

In some implementations, partial finger rings can be provided, e.g.,such that a portion of the user's finger is enclosed by the ring and notthe entire finger (e.g., a curved extension 1414 as described below). Insome implementations or uses, the user can grasp the outside of theextension member 1306 without inserting fingers in one or more of theapertures 1312 to, e.g., allow greater range of motion (e.g., fingertipmotion) of the controller for particular tasks. In some implementations,one or more input controls can be provided on the extension member 1306(and/or other extension members 1406 and 1506), e.g., which areaccessible to fingers that have been inserted in apertures 1312 duringuser of the hand controller.

In some implementations, a connection between the central portion1302/grip members 1304 and the extension member 1306 can be a directconnection (e.g., no rotary or other mechanical coupling), where theconnector can be made of a flexible material. The flexibility of such aconnection can be used to allow respective movement between the centralportion/grip members and the extension member, e.g., without use of amechanical coupling.

In some implementations, the extension member 1306 (and other extensionmember implementations described herein) can be made of a flexiblematerial, e.g., a foam rubber material, plastic material, or othermaterial allowing flexure and/or compression. This can allow respectivemovement between central portion 1302/grip members 1304 and theextension member 1306 as described above. Furthermore, the flexibilityof such an extension member 1306 can accommodate a variety of fingersizes, and may allow the extension member to cling to or otherwiseengage the fingers of the hand to provide additional grasping securityfor the hand controller 1300.

FIG. 14 illustrates a second example of a hand controller 1400 thatincludes an extension member having a finger aperture. In this example,a portion of the hand controller 1400 that includes the central portion1402 and grip members 1404 is similar to the corresponding portionincluding components 1102 and 1104 in the hand controller 1100 of FIG.11 and the corresponding portion including components 1302 and 1304 ofFIG. 13. In some implementations, other components can be used for thecontrol portion 1402 and grip members 1404, e.g., similarly as inimplementations described above.

A control body includes the central portion 1402 and extension member1406. Extension member 1406 is coupled to the central portion 1402and/or grip members 1404 similarly as extension member 1306 of FIG. 13.For example, extension member 1406 can be rotatably coupled to aprotrusion 1408, e.g., by a rotary coupling, such that extension member1406 can be rotated about axis 1410 with respect to the central portion1402 and grip members 1404. In some implementations, the extensionmember 1406 can be coupled to the central portion 1402 and/or gripmembers 1404 by a rotary coupling that includes two magnets, or byanother type of rotary coupling, e.g., similar to the examples describedabove for FIG. 13.

Extension member 1406 includes a single finger aperture 1412 which canreceive a finger of the user similarly as described for extension member1306 of FIG. 3. In some implementations, the aperture is formed by afinger ring 1414 in or coupled to the extension member 1406. In thisexample, the finger aperture 1412 is positioned in the extension member1406 to receive the ring finger of the user's hand while the first twofingers and thumb of the hand manipulate the grip members 1404 andcentral portion 1402. A curved extension 1416 can be extended from oneside of the extension member 1406, e.g., approximately in a directionperpendicular to the extension member rotary axis 1410 and curving apart of the circumference to form a second aperture similar to thesecond aperture 1512 shown in FIG. 13. The curved extension 1416 can becontacted or cradled by, e.g., the user's fifth finger, next to thefourth finger that is positioned within the aperture 1412.

Other shapes and configurations of the extension 1416 can be provided inother implementations. In some implementations, the finger aperture 1412can be made large enough to receive two or more fingers of the hand. Forexample, aperture 1412 and curved extension 1416 can allow the handcontroller to engage the hand of the user and allow the user's hand tograsp the hand controller 1400 more effectively, e.g., by reducing thepossibility of inadvertent releases or drops of the hand controller bythe user.

FIG. 15 illustrates a second example of a hand controller 1500 that canbe used similarly to hand controller 200 and that includes an extensionmember having one or more finger apertures. In this example, a controlportion of the hand controller 1500 that includes the central portion1502 and grip members 1504 is similar to the corresponding components inthe hand controllers 1100, 1300, and 1400 described above. Othercomponents can be used for the central portion 1502 and grip members1504, e.g., similarly as in implementations described above.

Extension member 1506 is coupled to the control body 1502 and/or gripmembers 1504, e.g., coupled to a protrusion 1508 of the central portion1502 and/or grip members 1504. In this example, extension member 1506 iscoupled by a spherical joint (e.g., ball joint) 1514 that allows theextension member 1506 to be rotated about a point P of the joint 1514.For example, spherical joint 1514 can use magnets, a mechanicalconnection, or other connection to provide the coupling. The sphericaljoint 1514 couples the extension member 1506 to the protrusion 1508. Insome examples, a ball 1516 of the spherical joint 1514 is rigidlyattached to the extension member 1506 and is rotationally coupled to theprotrusion 1508, e.g., magnetically coupled to a magnet rigidly attachedto the central portion 1502. In some implementations, the oppositeconfiguration can be used.

The spherical joint 1514 allows more directions of motion of theextension member 1506 compared to the extension member 1306 of FIG. 13,and allows flexibility in positioning the hand controller 1500 in theuser's hand during manipulation of the hand controller 1500. In someimplementations, spherical joint 1514 can be provided with friction tocause the central portion 1502 and grip members 1504 to stay in theirpositions when the fingers of the user release their contact with thegrip members 1504, or the spherical joint can provide a looser coupling.In some implementations, the ball joint 1514 can be detachable to allowthe extension portion 1506 to be attached and removed from the centralportion 1502 and/or grip portions 1504. For example, attracting magnetscan be located on the ball joint 1516 and the protrusion 1508.

In other implementations, spherical joint 1514 can be a ball jointhaving a bearing stud and a socket in which the bearing stud can rotate.For example, the bearing stud can be coupled to either the extensionmember 1506 or the central portion 1502 in various implementations.

Extension member 1506 includes two finger apertures 1512 in the exampleshown, which can receive the user's finger for stability and security,and are similar to the apertures 1312 as described above for extensionmember 1306 of FIG. 13. In other implementations, a single fingeraperture 1512 and/or a curved extension can be provided for extensionmember 1506, similarly to the extension member 1406 of FIG. 14. In someimplementations, each of one or more of the finger apertures 1512 can bemade large enough to receive two or more fingers of the hand.

FIG. 16 is a perspective view of a user's hand holding and operating ahand controller 1500 described with respect to FIG. 15. Hand controller1500 is held such that the user's first finger 1602 is engaging gripmember 1504A, the user's third finger 1604 is engaging grip member1504B, and the user's second finger 1606 is free to engage or activateinput controls on the hand controller 1500 (input controls not shown inFIG. 16). In the position shown, the user's fourth finger 1608 isinserted in the finger aperture 1512A of the extension member 1506, andthe user's fifth finger 1610 is inserted in the finger aperture 1512B ofthe extension member 1506.

In operation, the users fingers 1602 and 1604 can push or pinch the gripmembers 1504 toward the central portion 1502 to a closed position, andthe grip members 1504 can be restored to their open position away fromthe central portion 1502 via a restoring force, e.g., provided by theflexure of the grip member unitary member, and/or a spring or otheractuator. The extension member 1506 can be swiveled or pivoted in avariety of directions or degrees of freedom with respect to the centralportion 1502 and grip members 1504 using the spherical joint 1514.

In other implementations, the hand controller 1300 of FIG. 3 and thehand controller 1400 of FIG. 4 can be similarly grasped and operated bythe hand of a user.

FIG. 17 is a schematic diagram of an example controller system 1700 thatis mechanically grounded, and which can be used with one or morefeatures described herein for a master controller. Controller system1700 includes a control portion 1702 that can be engaged by a user'shand. The control portion 1702 includes a hand controller portion thatcan include one or more features described herein, as well as one ormore mechanisms. Some examples of control portion 1702 are described ingreater detail below with respect to FIG. 18.

Control portion 1702 is coupled to a serial kinematic chain 1704. Theproximal end 1706 of the chain 1702 is mechanically grounded. In thisexample, the kinematic chain 1706 includes three members 1708, 1710, and1712 that are rotatably coupled to one or more other members of thechain 1706 by rotational couplings having rotational axes. For example,member 1708 is mechanically grounded at a first end 1706 of member 1708and is rotatably coupled to member 1710 at a second end of member 1708.Member 1710 is rotatably coupled to member 1708 at a first end of member1710 and rotatably coupled to member 1712 at a second end of member1710. Member 1712 is rotatably coupled to member 1710 at a first end ofmember 1712 and coupled (e.g., rotatably coupled) to control portion1702 at a second end of the member 1712. The rotational axes of thechain 1704 can be sensed and/or driven by sensors and/or actuators. Someimplementations can provide additional actuated and/or sensed motion ofthe kinematic chain, e.g., about axes extending lengthwise through oneor more members 1708, 1710, and 1712.

FIG. 18 is a perspective view of an example control portion 1800 that ismechanically grounded and can be engaged by a user. In some examples,control portion 1800 can be the control portion 1702 of the controllersystem 1700 of FIG. 17. In some implementations, control portion 1800can be coupled to a different kinematic chain or other structure that ismechanically grounded.

In this example, control portion 1800 includes members of a serialkinematic chain 1801 that includes three members 1802, 1804, and 1806that are rotatably coupled to one or more other members of the chain1801 by rotational couplings having rotational axes.

Control portion 1800 can be coupled by a rotational coupling at a firstend of member 1802 to the second end of member 1712 of the kinematicchain 1704, allowing rotation about axis 1803 between members 1712 and1802. Member 1802 is rotatably coupled to member 1804 at a second end ofmember 1802. Member 1804 is rotatably coupled to member 1802 at a firstend of member 1804 and rotatably coupled to member 1806 at a second endof member 1804. Member 1806 is rotatably coupled to member 1804 at afirst end of member 1806 and coupled (e.g., rotatably coupled) to a handcontroller portion 1808 at a second end of the member 1806. Therotational axes of the chain 1801 can be sensed and/or driven by sensorsand actuators.

Hand controller portion 1808 can include features which can be contactedby a user, e.g., a hand of a user. For example, a handle, extensionmember, grips, switches, and/or other features described herein, e.g.,with respect to FIGS. 2-16, can be provided on hand controller portion1808.

In some implementations, the hand controller portion 1808 is coupled ata distal end of a serial kinematic chain that includes members 1806,1804, 1802, 1712, 1710, and 1708, with the proximal end 1706 of thechain mechanically grounded. This provides a stable platform for the useof the hand controller portion 1808.

In some implementations, the kinematic chain 1801 forms a gimbalmechanism that allows the hand controller portion 1808 to be rotatedabout the rotational axes of the chain 1801, e.g., axes 1803, 1810,1812, and 1814. Hand controller portion 1808 can also be translated inat least three linear degrees of freedom allowed by the kinematic chainformed by kinematic chains 1704 and 1801.

Various kinematic chains, linkages, gimbal mechanisms, flexiblestructures, or combinations of two or more of these can be used with themechanically grounded hand controller in various implementations toprovide one or more degrees of freedom to the hand controller. Somefurther examples of linkages and/or gimbal mechanisms that can be usedwith hand controller portion 1808 are described in U.S. Pat. No.6,714,839 B2, incorporated herein by reference.

FIG. 19 is a flow diagram illustrating an example method 1900 foremploying a master hand controller including one or more featuresdescribed herein, according to some implementations. Method 1900 can,for example, be used with an example teleoperated system or othercontrol system in which the hand controller is a master controller thatcontrols a slave device. For example, in some implementations, the handcontroller is a mechanically ungrounded master controller, e.g., mastercontrol device 122 of FIG. 1, and method 1900 can be performed by acontrol circuit component of the master control device 122, e.g.,performed by control system 150. In some examples, the control circuitcan include one or more processors, e.g., microprocessors or othercontrol circuits, some examples of which are described below withreference to FIG. 21. A single master controller is referred to inmethod 1900 for explanatory purposes. The master controller can be, forexample, any of the controller implementations described herein.Multiple master controllers can be similarly processed as described inmethod 1900. Other implementations can use a controller having one ormore features described herein with other types of systems, e.g.,non-teleoperated systems, a virtual environment (e.g., medicalsimulation) having no physical slave device and/or no physical subjectinteracting with a physical slave device, etc.

In block 1902, a master-slave control relationship is establishedbetween the ungrounded hand controller and a slave device, such as aslave surgical device or instrument in some examples. In someimplementations, this can be entering a controlling mode of the handcontroller. For example, this control relationship can be established inresponse to receiving a control signal from the hand controller or adifferent component of the system that indicates that the handcontroller is to enter a controlling mode (or following mode). In thecontrolling mode (during the established control relationship),particular manipulations of the hand controller cause changes inassociated state(s) or activations of associated functions of thecontrolled slave device. For example, motion of the hand controller inspace causes corresponding motion of a controlled instrument of theslave device, and/or activation of one or more controls on the handcontroller causes activation of a function of the controlled instrument(e.g., application of energy to a target site).

In block 1904, the control relationship established in block 1902 ismaintained. Two control options for the hand controller that aredescribed in this example method are moving the hand controller from apalm-engaged position to a palm-disengaged position, and moving the handcontroller from a palm-disengaged position to a palm-engaged position.These control options can be combined with various other motions of thehand controller in space. Other control options for the hand controlleras described throughout this description.

If the hand controller is in a palm-engaged position, then the methodcontinues to block 1906. In the palm-engaged position, an extensionmember at the proximal end of the hand controller, such as extensionmember 220 of FIG. 2, is engaged by (e.g., grounded to) the user's palm,e.g., is contacting, cradled, or cupped by the palm of the user's hand.

In block 1906, a first movement of the hand controller is sensed, fromthe palm-engaged position to a palm-disengaged position. The sensing canbe performed using any of a variety of sensing systems as describedabove. The palm disengaged position is a position of the hand controllerin which the extension member is not contacted or cradled by the palm ofthe user's hand. For example, this position can be approximately lateralto the palm-engaged position after a rotation of the extension memberresulting from the user pivoting the hand controller with fingertipcontrol, e.g., around a pivot point at or near the grips of the handcontroller. A palm-disengaged position can be to either side of thepalm-engaged position, for example. Some examples of a disengagedposition are described above with reference to FIGS. 6 and 7. The methodcontinues to block 1908.

In block 1908, a change is caused in associated operation of thecontrolled slave device based on the sensed first movement. For example,the change in operation can be motion of the controlled slave devicethat corresponds to the first movement of the hand controller. In someexamples, a slave instrument can be moved in a corresponding directionand/or a corresponding distance to the first movement in space. In otherexamples, a change in operation of an instrument end effector isperformed, e.g., jaws of a grasping device can be opened or closed incorrespondence with the first movement. The method continues to block1914.

If the hand controller is not in the palm-engaged position in block1904, the method continues to block 1910. In block 1910, a secondmovement of the hand controller is sensed, from the palm-disengagedposition to a palm-engaged position. This block can be performedsimilarly to block 1906. The method continues to block 1912.

In block 1912, a change is caused in associated operation of thecontrolled slave device based on the sensed second movement. In someexamples, the change in operation can be motion of the controlled slavedevice that corresponds to the second movement of the hand controller.In some examples, a slave instrument can be moved in a correspondingdirection and/or a corresponding distance to the second movement inspace. In other examples, operation of an end effector of a slaveinstrument can be changed, e.g., jaws of grasping device opened orclosed in correspondence with the second movement. The method continuesto block 1914.

In block 1914, it is checked whether the controlling mode is exited. Forexample, controlling mode can be exited in response to the useractivating an input control (e.g., a button) of the hand controller,removing one or more fingers from the grips of the controller, a voicecommand, other user input, etc. In other examples, controlling mode canbe exited in response to a procedure (e.g., surgical procedure) beingcompleted, a condition in the procedure (e.g., an unsafe movement orposition of the slave device occurs), etc.

If controlling mode is not exited, the method returns to block 1904 tomaintain the control relationship between the hand controller and theslave device. If controlling mode is exited, then in block 1916 themaster-slave control relationship is removed or ended, and anon-controlling mode can be entered by the hand controller and controlsystem in which manipulations of the hand controller do not controlfunctions or operations of the slave device. In some implementations,non-controlling mode removes physical or motion control of the slavedevice or other particular functions, while some functions of the slavedevice may still be controlled by the hand controller (e.g., input to adisplayed user interface of the slave device, causing output of audio,etc.). The controlling mode can be again entered similarly as describedfor block 1902.

The blocks and operations described in the methods disclosed herein canbe performed in a different order than shown and/or simultaneously(partially or completely) with other blocks and operations, whereappropriate. Some blocks and operations can be performed for one portionof data and later performed again, e.g., for another portion of data.Not all of the described blocks and operations need be performed invarious implementations. In some implementations, blocks and operationscan be performed multiple times, in a different order, and/or atdifferent times in the methods.

In various implementations, input controls of the hand controller can bemanipulated by the user's hand to provide control signals to the controlsystem and/or to the slave device, e.g., at any time during method 1900.As described above, such input controls can include grips and gripmembers, buttons, wheels, etc.

In some additional examples, input controls can provide control signalsto provide input to a displayed user interface, virtual environment, orother display provided by a display device, e.g., a user interfacedisplayed on a display 126 of FIG. 1. In some examples, sliding fingerswitch 230 (or other input control on the various hand controllerimplementations) can be a clutch control, where, for example, pullingthe switch toward a position toward the proximal end of the handcontroller (and/or maintaining the switch at that position) provides aclutch function to enter controlling mode (following mode) with the handcontroller. In some implementations, pushing the switch 230 to aposition closer to the distal end of the hand controller can exit thehand controller from controlling mode and cause it to enternon-controlling mode, or cause activation of a different function (e.g.,camera control mode in which motion of the hand controller controls acamera of the slave device, a user interface mode in which input fromthe input controls is provided to a displayed user interface, etc.).

Movement and orientation of the hand controller and activation of inputcontrols are sensed by various sensors as described above, and sensorsignals are sent to a controller (e.g., control system 150) in responseto the sensing. The controller activates one or more selected functionsof a plurality of functions provided by a system in communication withthe hand controller. For example, a control system 150 or control modulecan send commands to other system components to activate one or morefunctions based on the sensor signals received from the hand controller.

The term “function” as used herein can include one or more actions oroutputs (including operations or motions) of a controlled slave device,e.g., a surgical slave device. For example, a surgical slave device mayinclude surgical instruments as described above, and a function caninclude one instrument action or multiple instrument actions (e.g.,actions performed serially and/or at least partially in parallel). Insome implementations, a function can be a category of actions performedby a surgical instrument. In some examples, a cutting tool such as aknife or a surgical scissors may perform various actions in the categoryof cutting. In some implementations, the input control activating afunction causes one or more actions associated with the activatedfunction to be performed. For example, a cutting function can includeone or more actions such as moving a scalpel to create an incision in asurgical site with a straight cut. Alternatively, the cutting functioncan include actions such as snipping a blood vessel with a surgicalscissors, to be cauterized.

Surgical instruments may include cutting tools, grasping tools,cauterizing tools, irrigation tools, suction tools, absorbing tools,etc. In some implementations, the hand controller (or control system)outputs teleoperation control signals based on the sensor signals tocontrol functions including movements of the surgical instruments,and/or mechanical arms holding the surgical instruments, incommunication with the hand controller. Various functions can beassociated with such controlled instruments or tools, includingirrigation (injecting a liquid into or onto a surgical site or otherlocation), suction (removing of such liquid), clutch (disengage controlof slave device manipulator arms, e.g., to allow master controllers tobe repositioned without such control), turning on or off a camera(capture or record a scene at a physical location such as a surgicalsite), outputting energy by a cutting tool to cut or seal biologicaltissue, etc.

In some implementations, an input control may be activated by the user(e.g., button pressed) to cause a control signal to be sent and causeactivation of a function associated with the input control. In someimplementations, the input control is operative to maintain output ofthe control signal to the system while the input control continues to beactivated based on continued user input at the input control (e.g., abutton is required to continue to be pressed in order to maintain outputof the control signal to the system). In some implementations, themaintained output of the control signal causes the selected function tocontinue being activated by the system. For example, electrical energymay be applied to perform a coagulate function while an input controlbutton is pressed. In some implementations, an audio signal may beoutput by the control system to indicate the energy is being applied. Inanother example, a clutch function and non-controlling mode may beactivated and maintained while an input control button is pressed andmaintained in pressed state, while controlling mode is active while thebutton is released. In another example, camera control may be activatedas an input control button is continually pressed to allow the handcontroller to control camera position and/or orientation, and the buttonis released (deactivated) to return the hand controller back tocontrolling the position and/or grip of a surgical grasping instrumentand not control the camera position and orientation.

In some implementations, an input control on the hand controller can beused as a toggle to enter or exit control modes. For example, the inputcontrol button is pressed and released once to enter camera mode, and isagain pressed and released to return to instrument control mode. Inanother example, the input control can be used to toggle (swap orswitch) the arm or instrument being controlled by a hand controller,e.g., switch control to a different manipulator arm on a slave device.In some implementations, the input control may be used to deselectand/or deactivate a function, e.g. using a deselect toggle. In someimplementations, the input control can be used as a trigger to initiatea sequence of functions or actions, e.g., a staple sequence of a staplerinstrument.

In some implementations, a user interface (UI) and/or status readout canbe displayed on one or more display devices of the system (e.g., displayscreens, virtual reality or augmented reality headsets or goggles,etc.). The user interface can display information related to operationof the hand controller.

In some implementations, actuators can be included in the handcontroller to actively output forces on the hand controller, e.g.,motors, voice coils, etc. In some examples, such forces can be used toalert the user to particular conditions of the hand controller, of theprocedure, etc. For example, a vibration alert can be output by one ormore actuators of the hand controller (e.g., a motor rotating anoscillating element), where a vibration force is transmitted to the handoperating the hand controller. In some examples, the vibration alert canbe output in response to collisions that have occurred betweencontrolled slave instruments and other objects, in response to acontrolled instrument or arm reaching a limit to motion, as a safetyalert when using a cutting or energy-outputting instrument, etc. In someimplementations, distinct vibration signatures can be provided inassociation with different respective alerts (e.g., different vibrationfrequencies and/or amplitudes). Other types of forces can be used forsuch alerts in some implementations, e.g., single pulses of force, etc.

In some implementations, output such as haptic feedback on the handcontroller (e.g., on the grip members 204) and/or visual displays on adisplay device can be provided by the system to assist user operation ofthe teleoperated surgical system. For example, a user interface maydisplay warnings and/or error feedback on a display device, and/or audiooutput can be provided to indicates such warnings or errors. Suchfeedback can indicate functions that are potentially dangerous to apatient, and/or that a function to be activated is not appropriate(e.g., according to steps of a stored predetermined procedure) based onprevious hand controller movement or previous function(s) activated.

In various implementations, other types of computer-assistedteleoperated systems can be used with one or more hand controllerfeatures described herein, in addition to surgical systems. Suchteleoperated systems can include controlled slave devices of variousforms. For example, submersibles, bomb disposal units, industrialapplications, applications in hostile environments and worksites (e.g.,due to weather, temperature, pressure, radiation, or other conditions),general robotics applications, and/or remote-control applications (e.g.,remote controlled vehicle or device with a first-person view), mayutilize teleoperated systems that include slave devices for sensorytransmission (conveyed visual, auditory, etc. experience), manipulationof work pieces or other physical tasks, etc., and may use mechanicallygrounded and/or ungrounded master controllers to remotely control theslave devices. Any such teleoperated systems can be used with thevarious hand controller features described herein.

FIG. 20 is a diagrammatic illustration of an example teleoperated slavedevice and patient site 2000 for an example teleoperated surgicalsystem, which can be used with one or more features disclosed hereinaccording to some implementations.

A manipulator slave device 2002 can be controlled by one or more mastercontrollers of a master control device. For example, one or more mastercontrol devices 122 as shown in FIG. 1 can be used to control slavedevice 2002, or one or more hand controller described herein. During asurgical procedure, the slave device can be positioned close to anoperating table and patient (or simulated patient) for surgery, where itcan remain stationary until a particular surgical procedure or stage ofa procedure is completed. Slave device 2002 can include one or more armassemblies 2014, 2016, and 2018. In some examples, each of these armassemblies may include a surgical instrument 2024, 2026, and 2028,respectively. Each surgical instrument can include a surgical endeffector, e.g., for treating tissue of the patient. An arm assembly 2020(or other arm assembly 2014, 2016, or 2018) can be configured to hold animage capturing device, e.g., an endoscope 2030, camera, or the like,which can capture images depicting a surgical site or portion thereof.The endoscope can be in communication with to one or more displaydevices and transmit images to the display devices, such as displaydevice 126 of FIG. 1, a display device 2032 coupled to the slave device,and/or other display devices.

In this example, the arm assemblies may be caused to move and articulatethe surgical instruments in response to manipulation of the mastercontroller(s). This enables the user to direct surgical procedures atinternal surgical sites through minimally invasive surgical apertures.For example, one or more actuators coupled to the arm assemblies canoutput force to cause links or other portions of the arm assemblies tomove in particular degrees of freedom in response to control signalsprovided by the master controllers. The master controllers can be usedwithin a room (e.g., an operating room) that also houses the slavedevice and worksite (e.g., within or outside a sterile surgical fieldclose to an operating table), or can be positioned more remotely fromthe slave device, e.g., at a different room, building, or other locationthan the slave device.

Some implementations of the teleoperated system can provide differentmodes of operation. In some examples, in a non-controlling mode (e.g.,safe mode) of the teleoperated system, the controlled motion ofmanipulator slave device 2002 is disconnected from the mastercontrollers of the workstation in a disconnected configuration, suchthat movement and other manipulation of the master controls does notcause motion of the manipulator slave device. In a controlling mode ofthe teleoperated system (e.g., following mode), the motion of themanipulator slave device can be controlled by the master controllerssuch that movement and other manipulation of the master controllerscauses motion of the manipulator slave device, e.g., during a surgicalprocedure.

In some implementations, the teleoperated surgical system can include asupport on which a user, e.g., an operator such as a surgeon, can resthis or her forearms while gripping two grounded master controllers. Forexample, the master controllers can be positioned in a workspacedisposed inwardly toward a patient, beyond the support.

Features disclosed herein may be implemented in various ways, includingteleoperated and, if applicable, non-teleoperated (e.g.,locally-controlled) implementations. Implementations on da Vinci®Surgical Systems are merely exemplary and are not to be considered aslimiting the scope of the features disclosed herein. For example,different types of teleoperated systems having slave devices atworksites can make use of actuated controlled features described herein.Non-teleoperated systems can also use features described herein.

In some implementations, a controlled slave manipulator device can be avirtual representation of device, e.g., presented in a graphicalsimulation provided by a computing device coupled to the teleoperatedsystem 2000. For example, a user can manipulate hand master controllersand foot controller(s) to control a displayed representation of an endeffector in virtual space of the simulation and control virtualfunctions of the representation (or other virtual instruments) similarlyas if the end effector were a physical object coupled to a physicalslave device. Such environments can be used for training surgeons in theuse of the hand controllers, in some implementations. In some examples,the user can use or manipulate a master controller to control a proxyvisual (e.g., a virtual instrument displayed in a virtual displayedenvironment, and/or a virtual camera or physical camera included on theslave device or other device), and to control teleoperated surgical arms2014, 2016, 2018, and 2020.

FIG. 21 is a block diagram of an example master-slave system 2100, whichcan be used for one or more implementations described herein. As shown,system 2100 includes a master device 2102 that a user may manipulate inorder to control a slave device 2104 in communication with the masterdevice 2102. More generally, master device block 2102 can include one ormore of various types of devices providing one or more hand controllersthat can be physically manipulated by a user. For example, master device2102 can include a system of one or more master controllers such as oneor more hand controllers (e.g., master control devices 122 or variousother hand controllers described herein).

Master device 2102 generates control signals C1 to Cx indicatingpositions and orientations, states, and/or changes of one or morecontrollers in their degrees of freedom. For example, the master device2102 can generate control signals indicating selection of input controlssuch as physical buttons, hand controller states, and othermanipulations of the hand controller by the user.

A control system 2110 can be included in the master device 2102, in theslave device 2104, or in a separate device, e.g., an intermediary devicecommunicatively connected between master device 2102 and slave device2104. In some implementations, the control system 2110 can bedistributed among multiple of these devices. Control system 2110receives control signals C1 to Cx and generates actuation signals A1 toAy, which are sent to slave device 2104. Control system 2110 can alsoreceive sensor signals B1 to By from the slave device 2104 that indicatepositions and orientations, states, and/or changes of various slavecomponents (e.g., manipulator arm elements). Control system 2110 caninclude general components such as a processor 2112, memory 2114, andinterface hardware 2116 and 2118 such as a master interface and a slaveinterface for communication with master device 2102 and slave device2104, respectively. Processor 2112 can execute program code and controlbasic operations of the system 2100, and can include one or moreprocessors of various types, including microprocessors, applicationspecific integrated circuits (ASICs), and other electronic circuits.Memory 2114 can store instructions for execution by the processor andcan include any suitable processor-readable storage medium, e.g., randomaccess memory (RAM), read-only memory (ROM), Electrical ErasableRead-only Memory (EEPROM), Flash memory, etc. Various other input andoutput devices can also be coupled to the control system 2110, e.g., oneor more displays 2120.

In this example, control system 2110 includes a mode control module2140, a controlling mode module 2150, and a non-controlling mode module2160. Other implementations can use other modules, e.g., a force outputcontrol module, sensor input signal module, etc. As used herein, theterm “module” can refer to a combination of hardware (e.g., a processorsuch as an integrated circuit or other circuitry) and software (e.g.,machine or processor executable instructions, commands, or code such asfirmware, programming, or object code). A combination of hardware andsoftware can include hardware only (i.e., a hardware element with nosoftware elements), software hosted by hardware (e.g., software that isstored at a memory and executed or interpreted by or at a processor), ora combination of hardware and software hosted at hardware. In someimplementations, the modules 2140, 2150, and 2160 can be implementedusing the processor 2112 and memory 2114, e.g., program instructionsstored in memory 2114 and/or other memory or storage devices connectedto control system 2110.

Mode control module 2140 can detect when a user initiates a controllingmode and a non-controlling mode of the system, e.g., by user selectionof controls, sensing a presence of a user using a master controller,sensing required manipulation of a master controller, etc. The modecontrol module can set the controlling mode or a non-controlling mode ofthe control system 2110 based on one or more control signals C1 to Cx.For example, mode control module 2140 may activate controlling modeoperation if user detection module detects that a user is in properposition for use of the master controller(s) and that signals (e.g., oneor more signals C1 to Cx) indicate the user has contacted the mastercontroller(s). The mode control module 2140 may disable controlling modeif no user touch is detected on the master controller(s) and/or if auser is not in proper position for use of the master controller(s). Forexample, the mode control module 2140 can inform control system 710 orsend information directly to controlling mode module 2150 to prevent thecontrolling mode module 2150 from generating actuation signals A1 to Anthat move slave device 2104.

In some implementations, controlling mode module 2150 may be used tocontrol a controlling mode of control system 2110. Controlling modemodule 2150 can receive control signals C1 to Cx and can generateactuation signals A1 to Ay that control actuators of the slave device2104 and cause it to follow the movement of master device 2102, e.g., sothat the movements of slave device 2104 correspond to a mapping of themovements of master device 2102. Controlling mode module 2150 can beimplemented using conventional techniques.

In some implementations, controlling mode module 2150 can also be usedto control forces on the controller(s) of the master device 2102 asdescribed herein, e.g., forces output on one or more components of themaster controllers, e.g., hand grip members, using one or more controlsignals D1 to Dx output to actuator(s) used to apply forces to thecomponents. For example, one or more of control signals D1 to Dx can beoutput to one or more actuators configured to output forces to one ormore hand controllers, actuators configured to output forces on linkscoupled to a master controller (if it is a mechanically grounded mastercontroller), etc. In some examples, control signals D1 to Dx can be usedto provide haptic feedback, gravity compensation, etc.

In some implementations, a non-controlling mode module 2160 may be usedto control a non-controlling mode of system 2100. In the non-controllingmode, user manipulations of master device 2102 have no effect on themovement of one or more components of slave 2104. In some examples,non-controlling mode may be used when a portion of slave 2104, e.g., aslave arm assembly, is not being controlled by master device 2102, butrather is floating in space and may be manually moved. Fornon-controlling mode, non-controlling mode module 2160 may allowactuator systems in the slave 2104 to be freewheeling or may generateactuation signals A1 to An, for example, to allow motors in an arm tosupport the expected weight of the arm against gravity, where brakes inthe arm are not engaged and permit manual movement of the arm. Forexample, in a medical procedure, non-controlling mode may allow asurgical side assistant to easily manipulate and reposition an arm orother slave component relative to a patient or directly make some otherclinically appropriate adjustment of the arm or slave component.

In some implementations, non-controlling mode can include one or moreother operating modes of the control system 2110. For example, anon-controlling mode can be a selection mode in which movement of themaster controller in one or more of its degrees of freedom and/orselection of controls of the master controller can control selection ofdisplayed options, e.g., in a graphical user interface displayed bydisplay 2120 and/or other display device. A viewing mode can allowmovement of the master controller(s) to control a display provided fromimaging devices (e.g., cameras), or movement of imaging devices, thatmay not be included in the slave device 2104. Control signals C1 to Cxcan be used by the non-controlling mode module 2160 to control suchelements (e.g., cursor, views, etc.) and control signals D1 to Dx can bedetermined by the non-controlling mode module to cause output of forceson the master controller(s) during such non-controlling modes, e.g., toindicate to the user interactions or events occurring during such modes.

Implementations described herein may be implemented, at least in part,by computer program instructions or code, which can be executed on acomputer. For example, the code may be implemented by one or moredigital processors (e.g., microprocessors or other processingcircuitry). Instructions can be stored on a computer program productincluding a non-transitory computer readable medium (e.g., storagemedium), where the computer readable medium can include a magnetic,optical, electromagnetic, or semiconductor storage medium includingsemiconductor or solid state memory, magnetic tape, a removable computerdiskette, a random access memory (RAM), a read-only memory (ROM), flashmemory, a rigid magnetic disk, an optical disk, a memory card, asolid-state memory drive, etc. The media may be or be included in aserver or other device connected to a network such as the Internet thatprovides for the downloading of data and executable instructions.Alternatively, implementations can be in hardware (logic gates, etc.),or in a combination of hardware and software. Example hardware can beprogrammable processors (e.g. Field-Programmable Gate Array (FPGA),Complex Programmable Logic Device), general-purpose processors, graphicsprocessors, Application Specific Integrated Circuits (ASICs), and thelike.

Implementations provide various benefits. For example, a hand controllerdescribed herein can be provided with control over operation andfunctions of a slave device, such as a surgical slave device. Describedfeatures such as the length, shape, grips, input controls, and otherfeatures of an extension member of the hand controller provideadditional security and reduced fatigue in operating the hand controllerto reduce incidences of inadvertent slippage or dropping of the handcontroller by the user during operation. For example, the extensionmember is provided at a length and with a surface allowing the proximalend of the controller to be readily grasped and contacted by the palm ofthe user's hand, yet allows the controller to have large fingertip rangeof motion to provide accurate and precise control over slaveinstruments. The features increasing grasping security and reducingfatigue are of high importance in procedures such as medical proceduresin which controlled surgical instruments operate on a live patient. Dueto the fatigue that surgeons may experience over an extended surgicaloperation using master controllers, the described controller featuresare useful in performing teleoperated surgical procedures and otherprocedures or tasks.

Note that the functional blocks, operations, features, methods, devices,and systems described in the present disclosure may be integrated ordivided into different combinations of systems, devices, and functionalblocks as would be known to those skilled in the art.

Although the present implementations have been described in accordancewith the examples shown, one of ordinary skill in the art will readilyrecognize that there can be variations to the implementations and thosevariations would be within the scope of the present disclosure.Accordingly, many modifications may be made by one of ordinary skill inthe art without departing from the scope of the appended claims.

1. A control device comprising: a control body comprising a proximalend, a longitudinal axis through the proximal end, and a length; a thumbgrip portion coupled to the control body; a finger grip portion coupledto the control body; a ring element coupled to the control body andsurrounding the longitudinal axis of the control body; and an activationsensor; wherein the length of the control body is sufficient to engagethe proximal end of the control body in a palm of a hand of a user whilethe thumb grip portion is engaged with a thumb of the hand of the userand the finger grip portion is engaged with a finger of the hand of theuser; wherein the ring element moves in a linear translation along thelongitudinal axis with reference to the control body; and wherein theactivation sensor is configured to detect the linear translation of thering element.
 2. The control device of claim 1, wherein: the controldevice is a surgical system control device configured to provide controlsignals to a surgical teleoperated system.
 3. The control device ofclaim 1, wherein: the control device comprises a sensor configured todetect at least one of a position and an orientation of the controldevice in a working environment of the control device.
 4. The controldevice of claim 1, wherein: the thumb grip portion comprises a thumbgrip member rotatably coupled to the control body; and the finger gripportion comprises a finger grip member rotatably coupled to the controlbody.
 5. The control device of claim 1, wherein: the control device ismechanically ungrounded.
 6. (canceled)
 7. The control device of claim 1,wherein: the proximal end of the control body comprises an extensionmember; the control device comprises a tethered connection coupled tothe extension member at a connection point; the extension member isrotatable about the longitudinal axis of the control body independentlyof the control body, the thumb grip portion, and the finger grip portionsuch that the connection point rotates about the longitudinal axis ofthe control body; the tethered connection extends from the extensionmember radially with reference to the longitudinal axis of the controlbody; and the tethered connection operationally couples the control bodyto a master control system.
 8. (canceled)
 9. (canceled)
 10. The controldevice of claim 1, wherein: the control body is configured to allow theproximal end of the control body to be moved between the thumb of thehand and an index finger of the hand while the hand engages the thumbgrip portion with the thumb of the hand and the finger grip portion withthe finger of the hand, and the control body is configured to allow theproximal end of the control body to be selectively engaged by anddisengaged from the palm of the hand while the hand engages the thumbgrip portion with the thumb of the hand and the finger grip portion withthe finger of the hand.
 11. (canceled)
 12. (canceled)
 13. The controldevice of claim 1, wherein: the proximal end has an axisymmetric shapewith respect to a longitudinal axis of the control body.
 14. (canceled)15. The control device of claim 1, wherein: the proximal end of thecontrol body comprises an extension member that extends asymmetricallyto one side of the longitudinal axis of the control body; the extensionmember is receptive to grasping by at least one of the palm of the handor at least one finger of the hand during operation of the mastercontrol device by the user; and the extension member comprises a fingeraperture receptive to the at least one finger of the hand. 16.(canceled)
 17. The control device of claim 1, wherein: the proximal endof the control body comprises an extension member that is rotatableabout and translatable along the longitudinal axis of the control bodyindependently of the control body, the thumb grip portion, and thefinger grip portion.
 18. The control device of claim 1, wherein: theactivation sensor is configured to output a control signal that is basedon a predefined position of the ring element in a linear degree offreedom along the longitudinal axis, wherein the predefined position ofthe ring element is one of two or more different available predefinedpositions of the ring element.
 19. The control device of claim 1,wherein: the control device comprises a spring return element coupled tothe ring element; and the spring return element urges the ring elementto translate along the longitudinal axis of the control body.
 20. Thecontrol device of claim 1, wherein: the activation sensor comprises aswitch; and the linear translation of the ring element activates theswitch.
 21. The control device of claim 1, wherein: the control devicecomprises an input control coupled to the proximal end of the controlbody; the input control is configured to detect a threshold amount ofcontact with a finger of the hand; and the input control is coupled to aportion of the proximal end of the control body extending asymmetricallyto one side of the longitudinal axis.
 22. (canceled)
 23. The controldevice of claim 1, wherein: the thumb grip portion comprises a thumbcontact surface; the finger grip portion comprises a finger contactsurface; the control device comprises a distal weighted element and aproximal weighted element; the control body comprises a distal endopposite the proximal end; the distal weighted element is positioned ata distal end of the control body; the proximal weighted element ispositioned at the proximal end of the control body; and the distalweighted element and the proximal weighted element are weighted toprovide a center of gravity between the finger contact surface and thethumb contact surface.
 24. A master control system comprising: a controldevice, a control unit in communication with the control device, and aslave unit in communication with the control unit; wherein the controldevice comprises a control body, a thumb grip portion coupled to thecontrol body, a finger grip portion coupled to the control body, a ringelement, and an activation sensor; wherein the control body comprises aproximal end, a longitudinal axis, and a length; wherein the ringelement surrounds the longitudinal axis of the control body and moves ina linear translation along the longitudinal axis; wherein the activationsensor is configured to detect the linear translation of the ringelement; wherein the length of the control body is sufficient to engagethe proximal end of the control body in a palm of a hand of a user in afirst position while the thumb grip portion is engaged with a thumb ofthe hand of the user and the finger grip portion is engaged with afinger of the hand of the user; wherein the length of the control bodyis sufficient to allow the proximal end of the control body to passbetween the thumb of the hand of the user and the finger of the hand ofthe user to a second position; wherein the control unit is configured toprovide control signals to the slave device while a master-slave controlrelationship is provided between the control device and the slavedevice; and wherein the master control system is configured to maintainthe master-slave control relationship while the user moves the controldevice from the first position to the second position.
 25. (canceled)26. The control system of claim 23 wherein: the control body isconfigured to allow the proximal end of the control body to be moved tothe second position between the thumb of the hand of the user and theindex finger of the hand of the user while the thumb of the hand engagesthe thumb grip portion and the finger of the hand engages the fingergrip portion; and the control body has a shape configured to engage theproximal end of the control body in the palm of a hand of the user whilethe thumb of the hand pinches the thumb grip portion and the finger ofthe hand pinches the finger grip portion.
 27. (canceled)
 28. (canceled)29. (canceled)
 30. The master control system of claim 23, wherein: theproximal end of the control body comprises an extension member; thecontrol device comprises a switch; the switch comprises a ring elementsurrounding the longitudinal axis of the control body and positionedbetween the extension member and the control body; and a lineartranslation of the ring element along the longitudinal axis of thecontrol body activates the switch.
 31. A method of operating ateleoperated system, comprising: establishing a master-slave controlrelationship between a master device and a slave instrument, wherein themaster device comprises a control body and a ring element, and whereinthe control body comprises a proximal end, a thumb grip portion, afinger grip portion, and a longitudinal axis; maintaining themaster-slave control relationship while a user performs a first movementof the master device, wherein the first movement comprises moving themaster device from a first position to a second position, wherein thefirst position comprises the proximal end of the control body engaged inthe palm of a hand of the user while the thumb grip portion is engagedwith the thumb of the hand and the finger grip portion is engaged with afinger of the hand, and the second position comprises the proximal endof the control body disengaged from the palm of the hand of the userwhile the thumb grip portion is engaged with the thumb of the hand andthe finger grip portion is engaged with the finger of the hand; andsensing a linear translation of the ring element along the longitudinalaxis of the control body in response to movement of the thumb gripportion and the finger grip portion with reference to each other. 32.The method of claim 31, wherein: the first movement comprises moving themaster device to the second position such that the proximal end of thecontrol body passes between the thumb of the hand and the index fingerof the hand while engaging the thumb grip portion is engaged with thethumb of the hand and the finger grip portion is engaged with the fingerof the hand.
 33. (canceled)
 34. (canceled)
 35. (canceled)